Scale By The Bay 2021 : Tim Condon, Distributed Tracing with Swift

Scale By The Bay 2021 : Tim Condon, Distributed Tracing with Swift

Recording: Scale By The Bay 2021 : Tim Condon, Distributed Tracing with Swift

[Music] thanks very much uh it's wonderful to be here um even virtually um and today i want to talk to you about distributed tracing in swift and i want to take you on a bit of a journey and show how we implemented tracing in swift applications on the server and how some of the upcoming changes in swift are going to make tracing and server-side swift development really really good and really really fun so a quick introduction as mentioned i run a company called broken hands and we do server-side swift training consultancy stuff um i'm part of the vapor core team and vaper is a web framework for writing apis and back-ends and web applications in swift um i also sit on the sswg which is the swift server work group and we're responsible for pushing swift and the interests of server swift on the server with the swift language and so we have members from apple and amazon and mongodb and vapor and we basically we meet every couple of weeks as a steering group to try and influence the language and make working with swift on the server a great um i'm experience server size 15 me at raywenslick.com uh so it's a tutorial site and so we have lots of tutorials and books and videos on server side swift on there um and as mentioned i organize uh server side.swift which is a conference on server-side swift and i run a number of meetups as well both in here in the uk and around the world so i'm completely unbiased when it comes to server-side swift as you can tell um but we should have a great talk anyway and you can follow me on twitter and discord and slack and github as xerox tim so most of the conferences where i talk i'm talking to mobile developers who will know what swift is but they won't necessarily know that it can be used for the server applications but i suspect many people here won't have really heard of swift at all so swift um is a language that was developed by apple uh and released in 2014. um it its aim for them was to replace objective c as the language of choice for writing applications for apple platforms but swift has kind of grander ambitions and its main kind of his aim is to be a language that can be used anywhere and everywhere um and as part of this push it was open sourced in 2016. uh so the start of 2016 um it was open sourced and tool chains and runtimes were released for a number of platforms and these days it supports ubuntu centos amazon linux and even windows and it's designed to be a like a modern language it's designed to kind of join the cohort of new languages that we're seeing come out alongside the likes of like row and gus go and rust and kotlin and swift has a number of really kind of cool benefits so it has some really really fantastic language features it's borrowed and tweaked and taken features and uh concepts from other languages and learned from kind of the mistakes of the past so it has things like optionals it has generics it has protocols and it makes a really great language to write in but also has like a really easy learning curve so it's really easy to get going swift also has fantastic language interrupt so it has first class support for interrupting with c code so if you need to put in a c library you can call that natively and there's also python kit for talking to python you can talk to objective c and there's ongoing work at the moment to make interacting with a c plus seamless without needing any shims and swift is designed to be a really really safe language one of its kind of key benefits and aims is to be a safe modern language and so in practice this means um it has great features for ensuring the safety of replication so i mentioned it has optionals and these are effectively null nullability checks at a compiler level and so if you write your code properly and it's quite difficult not to um things like null pointer exceptions just are a thing of the past they don't happen and runtime crashes are exceedingly rare because swift is uh strongly typed and statically typed which means that kind of whole sway those errors that you get in other languages just aren't possible which makes it a really really safe language and swift is also really really performant so swift has a fantastic memory model and it uses automatic reference counting for memory management which means that you don't have to worry about managing your own memory and there's no garbage collector so you can be kind of have predictable performance on how your application is going to perform because there's no garbage collector that's going to kick in at some point in the future and so you know how it's going to perform and it also has a really really low memory footprint so a typical server-side swift application is in the region of kind of tens of megabytes um for a server application and you compare this to kind of some of the other languages like javascript and php and we're talking orders of magnitude less memory required to run the same kind of code and many of my clients run on fargate on ecs on amazon aws and we can't actually get small enough instances for memory um for the kind of tasks that we need we generally tend to sit at like five or six percent of memory usage of the container and so it's a really really performant language when it comes to memory but it's also a really performant language when it comes to cpu usage so swift is a compiled language so it compiles down to native machine codes so there's no jvm or interpreter running which means that you can you can get really really high performance code written in a really nice language that's safe and kind of swift is kind of joining these modern languages as i said but it kind of fills a nice gap so if you look at some of the modern languages that we have these days you have rust and rust is a fantastic language that's really picking up traction especially like the systems programming world and it's being used to write operating systems and it's even making its way into the linux kernel but it has quite a complex memory model and you have to manually manage your memory and has a really steep learning curve on the other hand you have something like go which is fantastically simple it has a fantastic concurrency model and a really easy language to learn but it's almost too simple in some aspects it doesn't have things like generics yet so you end up finding that you duplicate a lot of your code and just to write kind of main normal programs and there's this kind of happy middle ground somewhere with this gap and i'm not the first person to kind of point this out there are plenty of people and i see this quite regularly on twitter of people saying there's this like gap between the two um that is right for a language to come in and disrupt and in my completely unbiased opinion uh i believe swift is that language it has a really great it has really great features um it's easy to learn and has really powerful features when you need them so let's talk about tracing uh so to give you a bit of a background um and talk about tracing uh implementing tracing in one of the client projects i've worked on so the project uh is a grocery store uh in new york um and when the pandemic hit they realized they needed some kind of online presence and the only online presence they had was a simple store with like a pdf you could download of some of the products they had in stock and they had a contact form that you could fill in to place an order to go and pick up and obviously this isn't a great experience so we built them an online store with an ios app and a website where you uh customers could browse view their products they could um see past orders they could make purchases um and we also had the dashboard where store managers could go in they could manage inventory and put sales on and see orders manage customers etc and we also built a app for the staff members who go around picking the orders for for customers so previously they'd have to walk around with a printout of an email or a contact form and go and find someone when someone's written two packs of cookies and work out what are the cookies they wanted and try and make it kind of the right decision and so we've built them this app and they can walk around scan products and see which products they need to pick from different departments um and it's made made their life a lot easier so this was launched at the end of last year december last year and it went down really well customers really pleased with it and they have a much better experience for ordering online and staff members are really pleased but as well it makes it much more productive we've seen picking times go down from an average about 45 minutes per order to less than 10 minutes so staff members are kind of four and a half times more productive which is great so when we were kind of talking to see what's next like what we're going to do now we've kind of shipped this great product we came up with a number of kind of new features so the first was to be able to support other stores um so southbound marketplace which is the grocery store isn't just one store there are a couple of stores around new york and we wanted to be able to support them all and we wanted to enable things like delivery so that customers could have their orders delivered directly to them rather than having to go and pick them up and as we were talking we kind of realized we've built a really strong platform or strong service and so why not just let other grocery stores come on board and use this so we've done that so we've launched my web which is kind of ramping up at the moment and it's basically a service where any grocery store in the u.s can sign up they we can ingest their inventory and they get a website for free they get ios app they can use and they pay a monthly service fee um and it's all done for them without having to do anything um so as part of this we're kind of adding more and more features so we have we need to add things like store customization so the website is um basically generated on the fly so it's the same app and the same code running underneath the hoods but we need we have customization uh hooks so that we can like change the logo change the title change the color branding etcetera and we need to make sure that they put data separated because we can't have potentially rival stores viewing each other sales data so we're building a quite complex multi-tenant application and then we wanted more features on top of this so we want to bring in some of the payment processing in-house so that we can expand to more payment processes and we also want to do things like integrating with point of sales terminals so that we can have a single app and single api that manages inventory and manages products and sales and orders and stuff like that and so we're adding all all of these features and more and more complexity and when you're building any large application complexity is inevitable there's nothing you can do about it it's just up to you to decide how you're going to deal with that complexity and i'm not going to go into any of the kind of details or different architectures you can use but for us we chose microservices um because it allowed us to split up our code and isolate parts of the system and make them a lot simpler and also reuse code for different potential uh products and services but we went into it with opennights we knew that choosing microservices has its trade-offs um and we knew that we could have had to do a lot because you can't just go and choose microservices there are things considerations that you need to take into account and the main one especially for us was observability and monitoring and so how do we debug problems across our distributed systems so with debugging and monitoring on um microservices and distributed services there are three main things that you really care about the first is logs and for us this is fine because we automatically ingest all the logs from the containers into um cloudwatch and a3s so that's not a problem um the second is metrics and again that's not a problem because vapor automatically emits metrics that can be consumed by uh kind of any back end that conforms to an interface um so we could emit to prometheus and use grafina for our metrics but the third is tracing and tracing presented a kind of unique opportunity or challenge for us um to implement because tracing and swift wasn't really a thing and no one else had done it so for those of you don't know what tracing is um i'll give give a quick uh overview and basically in a distributed system when you have a request come in it hits maybe an api gateway or a kind of top-level api and that will make a request to a downstream service so in our case a request to the order service and the order service will make requests to two different services say the user service and the item service and then finally the item service needs to make a request to store service and that's to get all the information it needs to return to this request but what do you do if you have an error how do you kind of trace track down this error and work out where it is so you might have to go through the logs and view all the different logs of the different systems to find out what's going on and this is quite painful frankly and it takes up a lot of time and it's difficult to kind of aggregate everything so the idea with distributed tracing is is exactly the same apart from when you get a request come in you create a trace id for that request and so that trace id is then passed down to downstream services so that you can see each trace and where each of the systems it's hit and find out exactly where it's gone wrong and that makes it really simple and easy to find your problems and debug issues so as i mentioned tracing and swift provided us kind of a unique opportunity um and as we were building our um application uh swift released the swift distributed tracing library and this is an api package for packages and libraries to implement and use um so that it provides a consistent api for any package to uh implement tracing without caring about what kind of back end tracing is going to so each package or library that you use or depend on can use this interface and then the application can choose which back end to emit the traces to whether that's jaeger or open telemetry or x-ray for instance it doesn't matter so for us to implement this we had to do three different parts we had to implement it in three different places in our um application and we had to for three different libraries so the first one is the async http client so this is an hdb client library um that's kind of foundational to switch on the server so lots and lots of packages and libraries use it and it's designed for sending request third party apis and obviously when you're writing distributed systems and microservices you use this a lot the second is vapor so vapor is the web framework that we use and all our services are built on and so we had to fork it to implement tracing so that it works when the incoming requests come in and the final one was soto so soto is a third-party sdk for working with the aws api it basically uses the same specs as all of the other first party apis uh first party sdks and implements every single api that api that aws offers so whether that's talking to dynamodb or ec2 or scs or sns it has a way of doing that so to implement this in async http clients uh there are a number of steps you need to take so this is some code that um we currently have for doing this and when a request comes in or when a request is about to be sent sorry we create a spam and that span gets a number of attributes like the url and the http method and then we execute the request and when the response comes back we record the response like the status code and then we end the span and we can return the task and then the important part with distributed tracing is making sure that downstream services know about that trace so with the hdb clients what we need to do is inject the headers uh or inject the span attributes into the headers so that downstream services can see this and see the trace id and see any of the attributes to make sure that downstream services know that this is part of a trace um otherwise it wouldn't work so this is the really important part and the next part was implementing it in vapour so vapor again is very simple we create a span when a request comes into the api or the server and we set a number attributes like the http version the method the url the host name etcetera etcetera and then what we do is we wait we hand off the request to a request handler and when that request handler comes back we set a number of other attributes on the span so it could be the status code the response body um or the responsibility length sorry um whether it errored or not uh how long it took and then that's our span and then we can omit that to our tracing systems and that contains our trace and the final part was implementing it in soto and again it follows a very similar pattern um when we're about to make a request we create a span and these are all child spans of the parent spans that are kind of up ahead in upstream we set a number of attributes and some of these are required by x-ray which is amazon's tracing system such as like the database name or it could be the service name and then we pass that on so in order for us to instrument our code we have to be able to make sure that those spans are passed down to subsequent requests and downstream services so a request um handled like this in this which is vapor code and swift where we make a request to an external api so in order to instrument this with tracing what we have to do is pass a context through to the request um and that's how tracing is linked up and this is very similar to things like go where you pass the context around and that's how we did it and we implemented tracing and it went really well um it worked fantastically for us for tracing down bugs you can see here's a trace from our test environment where we're getting all of the information about a particular order we can see which services it hits um we can see how long each service took to respond we can see the database that it hit we can see the database table and the request they actually made to the database and we can see how long each took so we hunted down some really kind of horrible performance bugs on certain requests using tracing because you can see that for instance one database call took 900 milliseconds to return so we can optimize that particular database call and make sure that our requests will work rather than just seeing that this particular request and have to manually find out why it's taking so long and here's another example of a request failure so we can see here that we're making a request to get a particular item for a store um and we can see that it hit the api the api was fine it then hit the store service the store service was fine and then hit the item service and dynamodb which is our database now returned a a correct uh happy response return to 200 okay but the service failed which means that we couldn't find the item in the database or the item wasn't in the data that was returned from the database and that enables us to track down this error really really quickly so this is how we've implemented uh tracing in swift up to this point and it's worked really well and we've been really happy with it however there are some changes coming to swift um and that have recently been introduced just introduced to swift that are gonna make uh tracing and server application development um really really powerful so over the last few kind of couple of years or year and a half or so swift has been busy building up a concurrency model so up until very very recently swift had no concurrency model really no kind of first class concurrency model anyway and so uh in swift 5.5 which was released in september um it introduced it introduced a number of concurrency concepts um that we can use and this is kind of going to take swift to the next level of serve development because it makes things really simple um and one of these is things like async awaits so we use async weight and here is an example of some code um that we had before async await was a thing so before async was async the wait was a thing uh in our code when we're writing asynchronous requests or asynchronous code we have to do everything with callbacks and you can see it's pretty gnarly because we have several places where we return something uh we're kind of in a pyramid of doom um because we have to kind of nest each callback so with async await this just becomes really really simple and the number of lines get cut in half um and it makes the code a lot easier to read so swift as part of kind of how swift works with async await it needs a way of modeling asynchronous work and in swift this is a task so a task in swift is a kind of self-contained piece of work that's going to run asynchronously it's not a thread um it's not a thread local and it doesn't have any of the issues that you would see with threads and unsetting them and setting them etc and as part of tasks um swift has also introduced a thing called task local values now task local values are a way of setting properties onto a task that can then be read by downstream child tasks so one of the main use cases for task local values is distributed tracing so we can set span ids and trace ids in a parent task and then sub task read that spam id and trace id and attributes without having to do anything and this solves a number of problems so if we go back to the code where our callback callback base code looked you can see that in order to instrument this code for tracing we had to pass the context and pass the context and pass the context in a context in the context and every point where we're making either database calls or api calls we have to pass this context and this is what go does and this is problematic because it pollutes your code and it also relies on the fact that the programmer makes has to remember to pass that context because they might not and they might forget to they might pass something else and you're also reliant on the library that you're using libraries or libraries packages you're using making sure that they handle that context correctly and don't override anything and this is a problem that you see in java quite a lot um because java has a real problem with unsetting context contexts because there's no structured way to do this so you have to manually do this when you exit the async scope and this causes issues even with things like logging with the request id because you have to make sure as a library developer that you don't overwrite anyone else's thread local data and as an application developer you're hoping that library developers do this and there's no structure way of doing it so this isn't even possible this kind of error isn't possible in task local by design um so in a task local you can set your data in a task but nothing else can write it in child tasks child tasks can read it but they can't override it and it's thread say it's type safe as well so this kind of wipes out all these kind of different errors that we see um with tracing and thread local issues thread local data in other languages so with task local values in order to instrument our code to make sure that the task local values are passed to child tasks we have to rewrite it to this and yeah there's nothing to do it's done automatically for you by the compiler you don't have to pass it anywhere as soon as you set it on a task it's available for downstream tasks to read and it makes it really really powerful because this wipes out all these areas of forgetting to pass it making sure things are consistent so in order to set task local values um they're quite simple you can define a task local value so in this case we're setting a table name for a database and then we just set that with the task local syntax as you can see here and it's asynchronous as well because it might be set somewhere in the task that's already doing something and then we can send our request and we don't have to pass it anywhere because that task local value is set and then in our application code so this could be in the um soto where we're kind of making that request we can read that information out and again it's type safe and you can have nested objects if you want so if you want to nest your span data you can do and it's very easy to read and it works really really well so you don't have to worry about it's not like extremely typed um it's really simple to read so this is one of a number of features in swift that are making server-side development really powerful and asynchronous code really powerful so swift has also introduced concept of actors um there's a thing called sendable which is a way of marking your data as thread safe and this is enforced by a compiler by the compiler so that your code has to be thread safe um so thread safe and data access issues again become a thing of the past um it's another class of errors that swift of wiping out by using the compiler and type safety um to ensure that things are safe so actors is a way of the actor model um swift has introduced recently um so it lines up with uh things like sendable to make sure that accessing data across different tasks in an asynchronous environment is safe and works really well so um if you haven't already checked out comrad's talk from yesterday i highly recommend you do conrad talked about how to do distributed systems with swift and as part of that uh announced swift's uh introduction of distributed actors so this is a a way of doing cross-process or cross-service requests and kind of synchronization and data synchronization that's type safe enforced at a compiler level so first class support for doing cross service across process data access um it's extremely powerful it's very difficult to kind of comprehend um but again it just shows how far swift's coming for use as a systems and server language so um that's all i have thank you very much i hope you've enjoyed the talk and i hope you've enjoyed listening um about how swift how we're using swift on the back end um and i believe we have some questions now so i think i jump into uh the spatial chat thing or can i do them here uh yep we still have a lot of time so if you want to do you have uh actually add a question but i'm going to ask the question from this chord first which is uh oh yeah now we're getting three questions good times all right so since you're part of the server side working group or sswg anyway uh do you have any update on the various efforts to make swift less reliant on external libraries the reason why they're asking is has to do with uh docker images remaining relatively heavy for a compelled language especially compared to like rust um so yeah so um swift's main problem at the moment is uh linking with lib icu um for doing things like utf-8 strings and stuff like that uh and dates and stuff um there is ongoing work to make this um less reliance um it's a long long term piece so we're talking years and even the dog is sad about it [Music] but if you compile your swift application in the docker container you can actually compile it as a statically link or statically link this with runtime released um and your docker images go down to about 100 megabytes so they're pretty small um if you strip out all the runs of the um tool chain and stuff um they're not kind of like micro micro small one day hopefully um but yeah it's an ongoing piece of work that will take a number of years cool and one question that i had just quickly is about sendable does that mean like immutable or synchronously accessed or what um it's a yeah so um it's asynchronously access so if you want to change the data you have to go through a um basically an asynchronous access pattern so you use and use the awake keyword because something else might be accessing it so that's how it does work and it enforces that thread safety so it sort of like wraps an object in like a lock box or something like that yeah but it's a very lightweight lock um and the kind of uh the compiler enforces the fact that you can't lock yourself or create a deadlock so that's done by deadlock rather done by the compiler rather than you have to make sure you're not dead locking yourself right great and so the final question actually is just for you to share any links to uh the discord room about anything that they can use to know more about this topic so yeah absolutely i'll put a load of links in to forums and swift server work group and vapor and stuff like that definitely awesome great well thank you very much for taking the time to chat with us i really appreciate it and uh this was a really interesting presentation for me especially not being so familiar with swift and i know that uh you know this distributed tracing is something that we use at work and is really valuable for a lot of people so i appreciate you taking the time to talk to us about this topic cool um i'll go hang out in the special chat place as well if anyone has any other questions yep [Music] you