SBTB 2019: Salar Rahmanian & John A. De Goes, Next-Level Diagnostics.... with ZIO

SBTB 2019: Salar Rahmanian & John A. De Goes, Next-Level Diagnostics.... with ZIO

Recording: SBTB 2019: Salar Rahmanian & John A. De Goes, Next-Level Diagnostics.... with ZIO

[Music] all right good morning I'm John digos and I love functional programming and I love static types and I love the Scala programming language and when we use all three of these things together we have an unprecedented ability to ship code that is more correct and more principled than ever before but that doesn't mean all is sunshine and rainbows because even when we do FP using static types in the Scala programming language when we deploy code into production sometimes things go wrong they go very wrong and in modern async and concurrent applications it's not always easy to figure out what's causing the problem and fix it so today in this talk what I want to talk to you about is how you can use a library called Zeo to make it easier than ever before to troubleshoot production issues in your modern async and concurrent applications and joining me will be my good friend Salar and co contributor on some of the features that we've recently added to zo to make troubleshooting async and concurrent applications easier than ever before hi everyone I'm excited to be here at scale by the bay alongside John like I said I am loud I don't really need a mic this is my first conference talk so please bear with me and apologies if I make any blue bloopers thank you first conference talk let's give a round of applause to Salar for stepping up here on stage that's the right way to welcome a first-time speaker thank you for doing that so today we're gonna start off by acknowledging the fact that when we troubleshoot issues production issues and our async and concurrent applications built using Scotland's future it's not always easy in fact it's the opposite of easy sometimes and then we will take a sort of whirlwind tour to Ziya which is a next-generation library that helps you build a sync and concurrent applications and then we'll look at some of that goodies built into Zeo to make it easier to troubleshoot production issues in your modern applications and then finally we'll wind up with a little summary and sort of further resources so if you got interested in some of the stuff we talked about today you'll know where to look to find more information so Moore's law died roughly 2005 that's when processors stopped getting faster but they didn't stop making CPUs with with more and more transistors that kept on going we started getting more and more cores so modern servers they have dozens of course in some cases and that made life really hard for us application developers because suddenly were thrust into a world where we have to do concurrent and parallel programming and all apps have to do that if they want to be efficient if they want to minimize latency and maximize throughput they have to be concurrent but it's not enough to just throw threads at any problem because threads actually are not that efficient and if you create thousands of threads and you have most of them just waiting on other threads to compute results then you're actually going to slow your application down and so to solve this problem we've switched over to this model of non blocking computation known as asynchronous programming that's efficient applications have to embrace 100% of all modern applications they have to be concurrent and they have to be asynchronous in order to maximize throughput maximize efficiency and minimize latency now John I totally have a binding to the fact that we need to use async and concurrency for all of our apps but whenever I have used async concurrency in production I have hit a number of problems that I've always had to troubleshoot when I make use of the future code in Scala all right FairPoint tell me more for example when things go wrong I get a stack trace like this with not a lot of useful information for me to help me yeah that's right so if you've ever used future and you've done a stack trace on one of the exceptions in there you actually don't see the chain of operations that led to the failure you see a bunch of stuff relating to the guts of future and you see the guts of the fork/join thread pool in there and lots of other stuff that's not relevant doesn't help you solve the problem and that's because future because it's built for async programming keeps on submitting work to the thread pool which starts off with a fresh stack trace so all that information on the provenance of your failure is erased now this is made worse by the fact that whenever I want to see what all my features are doing again I'd look at the thread dumps to find that information and again the information is not useful to me another good point because in synchronous applications right we when we want to figure out what's going on if we have like some part of our application that's stalled we just do a threat dump and we can see all the threads in the system and we can see their state we can see which ones are waiting on other threads there's actually no way to do anything like a future dump when you're using scoffs future because we need to a thread dump with a future based application you only see the futures that are actively running now you don't see the tens of thousands or hundreds of thousands of other futures that aren't running right now because they're waiting for someone to complete some promise somewhere and the other kind of errors that I thought that I would struggle with are when the futures are hanging or something is blocking it again I struggle to find the information about it right so in the world of synchronous programming if we have one thread that's waiting for another thread to release some sort of monitor then we can see that in the thread dump we can know why our application is hanging because we can see that in the thread dump however when we've got one future that's waiting on another future to do something we don't see that anywhere because there is no such thing as a future dump and not only that but with futures both of those futures can actually be garbage collected which means all evidence of the hing can just magically disappear from the heap so these are all legit prompts I'll give you the SLR these are all the JIT problems with Scotland's future when it comes time to building a sink and concurrent applications however future is kind of old at this point it's ten years old and we as developers in the scala community have had a lot of time in production experience to learn from the mistakes of future and build something better so now I'm going to talk to you a bit about a library called zo that's Lara and I are Co contributors to that helps you build modern async and concurrent applications in a way that's better than future so zo is built on a next-generation fiber based concurrency model now Don what is what is a fiber and how does it compare to a thread good question so threats JVM level threads are mapped one to one on two operating system level threats and threads are big heavyweight things that consumed a fixed amount of resources they have a pre allocated stack size and so forth they're very relatively slow and they consume a lot of resources and as a result a modern application can't actually scale past about ten thousand threads before the overhead of all the threads who starts eating into the performance of the application at unacceptable levels so Zeos fibers on the other hand are kind of like a lightweight green threads of execution they're super cheap and unlike normal threads zo fibers don't block there are a hundred percent asynchronous which means instead of blocking the thread to wait for some other fiber to compute something they just go to sleep and when they're going to sleep they don't actually consume any thread resources which means that unlike threads we can scale well past that ten thousand k limit we can get into hundreds of thousands or even millions of fibers without running into performance problems now my top three pain points with Scala futures are the fact that I can't cancel them I always need to have an implicit for the execution context and as I mentioned the stack traces and threatens provides useless information to me right so all these are real drawbacks with future and zo because it was built well more than eight years after Skylar's future had a chance to learn from some of the mistakes made with future and improve on it to build on the shoulders of future to result in a data type that has much more power in every way than future for example there is no passing of implicit execution context with zo all of that stuff is sort of passed for you in the background and doesn't limit your ability to decide where things run you just no longer need to pass all these implicit surround also zo gives you an ability to do precise parallelism future doesn't have that ability every time you call future dot apply that results in a submission to the thread pool so it's very easy to get into a situation where you have ten thousand threads spawned by mapping over a list and stuffing everything in the future doing something asynchronously zo always gives you precise control of the amount of parallelism with every single operation like you mentioned SIL our futures can't be canceled and that's a big problem because what happens is when you stop waiting for a future to be completed or when a bunch of features that you're sequencing or traversing one of them fails all the other features keep on running in the background consuming resources they consume network resources and memory resources and so forth it's not uncommon in a modern application have tons of threads sitting around executing future logic for results that are ultimately going to be thrown away and 0 on the other hand supports cancellation so when a resource is not needed it gets cancelled right away and everything is all resources are safely cleaned up stack traces like you mentioned are totally useless with future zo introduces something called an execution trace which is better than a stack trace gives you more information than a stack trace and it works across asynchronous regions in your code as well as parallel regions it's not really possible to refactor future based code because if you extract out some variables in a future based code then you might get a different ordering of the results in which those futures are run because futures an eager datatype and not only that but in some cases cases it will change the number of times those those things happen in the real world Co because it's based on a lazy data type you can refactor your code all day long without changing its behavior future because it's eager you actually can't write future based Combinator's for example you couldn't write a retry function that takes the future and returns another future that will do the effect of the first one a certain number of times until it succeeds you can't write that with future but because he has effects are lazy it's very easy to write that type of logic future has an air channel baked in but it's always fixed to be throwable and what that means is when you have a future you never know whether or not an air has been handled and so sometimes you're going to assume that an air has been handled in the future but you're gonna be wrong some air sneaked into that future and it's gonna blow up your application in an unexpected way that's because futures airs are dynamically typed the CIO's effects on the other hand have statically typed airs so you can describe whether or not effects fail and if they do fail how they fail and you can use the Scala compiler to enforce that future has no version of try finally that works across async operations zo gives you a version of try finally that can work across async and parallel regions of your codebase so that you can make sure if you acquire summary resource you definitely end up releasing it as you can build applications that don't leak resources future is a reasonable way to do a sync and concurrent programming however it doesn't provide you any tools that you need in order to build concurrent applications so for example you don't have any async version of Java's blocking cube or of a semaphore or of a lock or all the wonderful stuff in Java util concurrent future doesn't give you a sync versions of any of that Anzio on the other hand gives you an async version a high-performance async version of all the goodies you've come to know and love in the java.util concurrent world future is known for not so great performance and that's because by default that submits every operation to a thread pool which interferes with the way that modern CPUs get a lot of their performance which is by caching stuff in poor local caches and so when you submit every operation to a thread pool you end up basically starting from a bunch of cold caches and having to warm those caches up Zeo will batch by default more than 2000 operations on a single core before a fiber yields to other fibers and then finally future based applications are not easy to test but Co because it's based on pure functional programming at some a feature called late binding that makes it very very easy to test your Zeo based code in a deterministic fashion that runs quickly without actually interacting with the real world in this example you can see a hello world application with Zeo it extends EO app and our main function here called run gets to return an effect that it creates by calling put store line with a string hello world now John out of curiosity what does the dot fault why do we need it a good question so put stir line returns an effect that succeeds with a unit value but the run function requires we return an effect that succeeds with an integer status code which will be used as the exit code for the process when our main application terminates so what we do is we fold over both air and success case mapping the air case to the integer one and the success case to the integer zero to end up getting an effect out of that which succeeds with an integer value in this little example we can see an interactive application that prompts the user for their name reads it and then writes it out back to the user and you can see sequential flow here however notice we are not programming with statements you don't program with statements using Zeo you program with values and operations on values in this case we have three different effects and we're composing them together with flatmap how we do that in a more same fashion as we use for comprehension since kala and every for comprehension in scala you can your intuition for that should be hey this is somewhat like a statement but instead of doing side affecting eager operations we're just building a lazy effect out of this for comprehension now looking at the code John I see that my app is extending app do we get any extra batteries included with the app yeah so because the effects are totally lazy that means if you want to see what they do you have to run them and to run a Zeo effect you need a runtime system which bakes in the sort of core logic of the Zeo interpreter and it uses a thread pool and it has a an error reporter that's called with any of the errors in your application that aren't handled so you can log those and make sure you don't lose them and app zero dot actually bakes in that runtime system and and uses it to execute the effect that you returned from run in this example you can see how easy it is to define new Combinator's on Z you effects in this case I write a Combinator that allows you to take two tasks and return a new task that computes both of the other things in parallel you can see how easy it is to do this with a fiber based concurrency system you simply fork the left task to get a left fiber and then you fork the right task to get a right fiber and then you asynchronously join the left to get the a out of the left hand side and you asynchronously join the right to get the B out of the right-hand side and then at the bottom of that for comprehension you yield a tuple that contains the a and the B this is how easy it is to do parallelism with fork/join now john what will happen if one of the fibers fail good question so if something fails in the middle of that for comprehension if one of the fibers fails and hence the join fails then what's going to happen is the other fiber will be gracefully shut down instantaneously and all resources will be safely cleaned up and that happened because we're calling dot interrupts children on the effect return from the for comprehension in this bit of code we have this HTTP GET method that for a given URL gives us back a task of a response and maybe we have this particular URL that's kind of flaky and we need to retry it a bunch so what we could do is we could call this retry method on effect and we can pass it a schedule a retire policy that tells us how we want to retry that and we're gonna get back a new task a new effect that's going to execute the other one according to the specified retry policy in this case our retry policy does exponential fall off starting from 10 milliseconds and then going to 20 then 40 then 80 and so forth or once that reaches more than 60 seconds it's gonna fix spacing between the retries to 60 seconds and it's gonna do all of that up to a maximum of a hundred times and finally in this last example we're gonna use the same age to beget method to create a load tester and this load tester we're going to create quite easily we're actually going to do a for each loop over the integers from one to a thousand only instead of calling zeal for each we're gonna call the parallel version of that Co dot for each par which executes every iteration of the loop in parallel so in parallel we're going to be executing all of these things together doing HTTP gets for the same URL thus creating the load tester that's executing thousand concurrent requests and then if we wanted to do that repeatedly we would just add dot forever onto the end of that effect and that would put it in an infinite loop that keeps on pounding our server with a thousand requests per second or whatever so that's unfortunately all the time we have to show you about coz o is very rich library and I encourage you to check out other top talks on the topic for now though we're gonna move on to take a look at that sort of tool bag that Zeile gives you to troubleshoot production issues in async and concurrent applications now John before we carry on the Zeo provide anything for us to prevent problems in the face going to production it's a good question so Zeo gives us a few goodies that can help us prevent us from shipping issues that lead to production errors and one of them is testability so in in this screenshot here you can see an application that reads and writes from the console and sleeps and so forth and in it looks like this would be hard to test but because we're using Zeo which is based on pure functional programming and because Co has very very good support for late binding it allows us to test this deterministically using Zeo test so in this test were actually verifying that it's interacting with a console in an appropriate way and we're advancing time inside our tests this test will run deterministically and instantaneously without any interaction with the outside world and you can do that for any Z application you write you can write tests that run deterministically and instantaneously without interaction of any kind with the outside world also Future has this dynamically typed air channel which means that when you have a future you never know a compile time if that future is going to fail but what zo does is it takes that air channel out and six it in a type parameter in the zo effect type it's the second type parameter and so that gives you the possibility to plug in your own air type so if you want to use your own ADT your own data type for custom airs business airs domain airs you can do that you can also plug throwable in there if you want why not and then you have basically what you get with future but if you want to describe the facts that can but can not fail you do that by plugging nothing into that type and because in Scala there are no values of type nothing this represents a compile time guaranteed at this class of the facts with nothing as their air type cannot fail and when you have the ability to statically represent effects that cannot fail it helps you reason about with the help of the Scala compiler the error behavior of your application in this case we're using the Scala compiler to help us prove that this process defect cannot fail we call process upload and that can fail with some sort of blow air but then we call the or else operator on that effect to have a fall back and our fall bot back cannot fail and we know it by looking at its type and the Scala compiler knows it so we're able to prove at compile time that we've handled all errors in this chunk of code now John I've heard that with try and finally some errors get lost is that true yeah and not just with try finally but even with future these are lossy err models and let me give an example of where that happens let's say we're doing one thing and then we fork that off into two different threads now we have one on the top and one on the bottom and then let's say the thing on the top fails with an airy one and then let's say the thing on the bottom fails with an air e2 and as its failing it has a finalizar installed and let's say that finalizar is poorly behaved and that finalizar fails as well and what's gonna happen in this case well e one is the one that will get reported back it's the one will be allowed to catch in Java and Scala and it's the one you can recover from in the future e two on the other hand um gets thrown away entirely you're not gonna see that in any log file it's not even visible to your application and then the finalizar air gets reported at least with future it gets reported on a side channel so zo says now this is not a great state of affairs it introduces this new data type called cause which comes from functional programming it's a free semi ring and what it does is it allows lossless capture of a hundred percent of all errors in the events of a failing effect whether they're a sync or concurrent airs and so it captures the e one it captures the e two it captures the finalizer air as well inside this data structure and when you go to pretty print this you can actually see which airs occurred in parallel with which other errors and which occurred sequentially which airs our primary heirs and which errors are secondary errors you have a hundred percent of all that diagnostic information to figure out what went wrong now don't tell me how does this execution traces actually work yeah so execution trace is unlike stack traces future tell us every single line of code leading up to the failure not just every single method but every single line of code even in a four comprehension leading up to the line that failed and not only do they tell us what did happen in the past they also tell us what would have happened in the future had that line of code not failed and you can see here the amount of information we get out of these execution traces is tremendous and it allows us to figure out all the steps that led up to a given failure so we can troubleshoot and fix production issues faster way faster than we could with with future without any print lines early earlier I asked if and with ask our futures I can't see what all my features are doing with Zeo can I see what all my fibers are doing absolutely so zero is the only library out there that allows you to do a fiber dump which is the analog of a thread dump but for fiber based concurrency systems and this fiber dump which is modeled after a thread dump so if you've ever used a stack or something like that to print out all everything that's going on in your Scala application then you're going to be familiar with this format we can see that fiber number 17 which has been executing for 41 milliseconds is currently waiting on fiber number 14 to do something not only that but we can see that it's suspended it's currently interrupted also it could be canceled from the outside it has done zero async operations so far and we can see the exact line of code where it's waiting on fiber number 14 to complete and then in addition to that of course we get the full execution trace which contains the history of all operations leading up to the point where it was suspended not on does it solve my poem of being having visibility overhanging fibers yeah absolutely so remember when one future is hanging on another future there's no way to see that and so people insert lots of print lines to figure out why parts of their future based application just stop and it's a it's a huge problem even the print lines won't help you if those futures are garbage collected which they they probably will be in a lot of cases so what Zeo does is it actually tells you when you've got one fiber waiting on another fiber to be completed in this case what we do is we make a promise we hand that off that promise off to some other method and then we have another fiber that's sitting around waiting on the result of that promise and we can see inside the fiber dump we can see that that's fiber which we forked off in that line in the for comprehension is currently suspended on the fiber that created the promise allowing us to for the first time ever to diagnose quickly issues where our application part of our application just hangs because no one completed something and then another Zeo first is actually necessary because Co effects can be cancelled you can cancel them you can't cancel futures so futures don't have this problem but what happens in a Zeo application is a future or a Z effective stops executing because someone canceled it and sometimes that leads to a problem you're left wondering well who cancelled it and why maybe it wasn't supposed to be cancelled and you've got to figure that out so when ever and effect in Zeo is canceled Zeo will actually tell you which fiber it was that cancelled that effect so you can figure out why part of your application just was interrupted so as we've seen in this presentation when you go to troubleshoot an async concurrent application built on future it's not easy you run into all sorts of problems you run into problems because you can't do anything like a future dump you can't dump out all the futures running and see you know what their state is you can't do that you don't get any kind of useful stack trace out of future errors and then you have this issue where when futures hang on other futures there's no insight into that and all evidence of that hang can be garbage collected so you really are crippled when it comes time to troubleshoot issues with production applications built on scholars future zo however is a next generation take on future it's designed as a modern fabric on which to build a sink in concurrent applications and it takes the opportunity you know complete brand new design to learn from our 10 years of production experience with future to build something that's better and then it comes with especially as a result of some recent contributions that Salar and I have been working on it comes with a set of tools that no other fiber based concurrency system out there has these set of tools allow you to do fiber dumps so you can see all the state of all fibers running in your application they allow you to do execution traces which are on by default there's nothing you have to do to get that it's just out-of-the-box support for that and they also allow you to see when one effect is waiting on another you can see that information you also have lossless errors that never throw away any information so you never left wondering what happened because you've got some sort of secondary fail failure in the log file but the actual true cause of the problem was thrown away either by Javas try finally or Scala's try finally or by you know futures error error handling ski you get lossless air model and you also get statically typed airs which means that you can use the cout Scala compiler to help you get the error behavior of your application right before you ever deploy to production so if all this sounds interesting i and i encourage you to check out some of the others hawks that's i've done and other people have done about c or check out the website there's lots of great resources for learning more about how a library like co can help you build modern applications that are async and concurrent but in a way without all the drawbacks of future i want to give a special thanks to alexei fortune a don't see him here he was be off organizing this conference i know he worked very hard to bring us this this conference i want to thank him for inviting me in celerity to this presentation and also i'd like to announce that's early next year we're going to be doing a zo hackathon here in the Bay Area this is a great chance to come and hack on Zeos the zu e cursed ecosystem the broader zo ecosystem but also we have an unconference a to learn a lot about zo and the bigger ecosystem we've done two of them so far we've done one in Warsaw which was about sixty people and one in Berlin which is about 120 people and who knows how big the one in the bay area will be but I expect lots of good things from that so please look out for that and I'm sure Salar will announce it to the local meetup you can learn more at zio dev which has decent documentation but it's always in need of improvement full requests welcome and then we have a great discord community discord as brand new we've just been using it a few weeks but there's already more than 500 people on there and they're super helpful and eager to help out anyone who has any issues also Stack Overflow has a growing zo community to help out with any issues you might run into you can follow me on twitter @ JT goes you can sort support my work especially my work blogging and speaking at conferences on patreon if you want to get a few free goodies on the side and then you can read me read my blog at D ghostnet where I quite often blog about functional programming in Scala and feel free to follow me on Twitter and my blog as well I won't try to spell my name for you it's long we're hoping to have a lot of good content on what's happening in Ziya zo and of course the scala community here in the Bay Area as I run the meetup thank you all right thank you so much [Applause] I'll take questions in the hallway track since we're out of time [Music]