Futures: Twitter vs Scala

scale.bythebay.io: Vladimir Kostyukov, Futures: Twitter vs Scala

Recording: scale.bythebay.io: Vladimir Kostyukov, Futures: Twitter vs Scala

hello everyone that's so many people I'm actually surprised so many people interested in this boring topic so let's say my name is flat I'm on the cs LT matter which is the team that owns finagle and three refuges and this is my completely unbiased comparison of the Scala futures and the two air futures and but I have to warn you it's not it's not really a competition it's not like a bottle there's no winners or losers and I'm not gonna tell you that you should go ahead and use three futures for a next project well you should totally do that but not because I told you you should run your own tests and instead they'll try to give you some insights into why twitter futures exist how are they different from Scala futures and how those differences can serve a great good for networking libraries like finagle itself and before we start a little closer okay something like this better so and before we start talking about the differences between those two to future implementations let's look at the historic context of white where futures exist in the first place why they happened to be in the Scala ecosystem and which seems to be really confusing topic based on the internet discussions so let's scroll back for a second year 2010 what tools exactly did we had for doing a synchrony on the JVM and the answer is we didn't really have that much there was Java concurrency which isn't actually future but somewhat avoidable so you have in order to interact with its value you have to actually block on it and wait for for the value to be available which is not really good grade for asynchrony there is some Scala actors future which is not even a cure future it's Skala actors future it's the one that was building into the standard library and it had the map flatmap things Colback so it's more or less the future but it was highly coupled to the to their actors library so there's no way we could have used that Brittany dark times summer 2010 probably too early to talk about a synchrony and this kind of thing Scala was a 2.8 at that point it was like zero people using it in production and there was two and a half years before c14 happened which is known as Scala futures and promises Scala 2:10 and even more four years before actual Java future happened which is completely the future so they made an executive decision to roll out one thing which is known as to her future and thanks to the github history we can look into that and it's that simple it's less than 100 lines of code and it's not stacks a if it doesn't have all these fancy futures but it was like a good start you know it was something we couldn't take and iterate on it was a um was a great deal actually so alright so what happens next and a half years later we've got Scala futures in place you've got Scala to Tam Scala futures and scallop promises and yep as you know finagle and the rest of the tree rican system is still using its own futures its own future implementation and there are actually reasons to that multiple reasons I should say technical reasons because this is a technical conference we're not talking about business reasons of switching from one future implementation to another because there are nom there's no way you can tell 2010 engineers stop doing what they're doing and rewrite their services using UI synchrony primitive it's not gonna happen yep so we're just talking about technical reasons so those two future implementation at that time we are already so different in terms of compatibility and options you may have and they remain different until this very moment and you'll see later how we actually utilize the differences in libraries like finagle and the rest of the troika system so and the first biggest difference is probably scaling is something that most of the users interact if first when we started using three refuges and this parallel the most confusing part because as you know there is no execution context for Twitter futures there is nothing that implicitly common into map and flatmap functions so there is no actually an easy way to tell on reached at pool to executors giving callback and the more configured the most confusing part about that is twitter futures sort of prefer local execution they favor something called local scale learning so the way it works when you satisfy a promise it actually goes and runs all your callbacks from the same trap you used to satisfy this promise so it's kind of counterintuitive there is no asynchronous there is no parallelism how is it in the future but in fact if you think about that if you think about the history Twitter future if you think about finagle contacts to torts which it's biased there's there is actually this thing is like a shared threat pool with an everything angle application which actually used to round your promises within multiple threats and make sure to keep a good affinity between connections requests and your traps and the trade pools is that you satisfy and run your promises within multiple threats and do that in pretty much uniform mirrors so you got a good concurrency and parallelism out of the box so you don't have to do anything else so this technique is called local scaling and you can take a look at the scaler in the util library how it's implemented it's really simple it's just like again really less than 100 lines of code and it's just attach the local scalar to the current thread like thread local skiller in just runs your stuff it's very simple and it's not only makes it really easy to run and utilize your resources better because you don't have to spin out a new tab and you've had pools and stuff like that it actually helps you to reduce context switches as well and we've been experimenting in the past trying to swap the scalar from something like local scalar into the ax for joints killer which is the forum pool and it didn't actually work great because because of the burden of switching hats and doing context switches so this wing works pretty pretty well for us so the next is really exciting stuff I think is something called route compression and promise Lincoln and you can think about that as a tail call elimination for your future recursion let me just show you an example is it big enough like this alright let's see let's say you want to write a function that computes a factorial of a given number and we want to structure that in a way that looks like a tail recursive function so it takes an argument we wanted to accumulate the value on as a future thing and here and it returns that as a result so the first thing we do is check the terminal condition then return the death oh sorry this one already I could abrogate you see we can just return that and if not we just do the flat map here and we get the value out of it and this is I and we just call the same function again here we have to decrement the counter and we have to accumulate the result that value and we multiply that something like this and you know we can just run it here with initial value and it works it's not it's not a surprise that it works right so so what exactly happens here let's talk about that this function that actually produces a new promise on each iteration and the probably used that promise as an argument to the flat map function that is cold within that the same function sort of like future recursion and if you if you try to imagine how it looks like it's sort of like a chain of promises linked from each other's flat map callback sort of like a link list thing and what's so bad about this chain what so what's so terrible at this and the problem is those chains could be really long and the whole thing can not be garbage collected until we actually resolve the whole structure so you have to get to the bottom of the recursion you have to meet the terminal condition and the result the whole thing and then you can collect that this is effectively spacely because those chains might be really long and you can allocate some stuff within your promises your callbacks or even better is that those chains can be unbounded you can have infinite recursion and it actually works very well on finagle so and how to solve this problem there's actually a solution really nice really nice one similar to a tail call elimination for your functions for your stacks too that helps you to UM widen your stack promise link in the root compression helps you to unwinding your promise chain essentially instead of forming this long chain and linked list of promises you're trying to link every single individual promise back to the root 1 directly such that those temporary promises that you have them in your recursion can be collected as long as you leave this cope basically as long as you're not reference and I'm from the user skilled and this is really efficient and really really effective and we use that a lot in finagle here's for example be something called ping base filler detector that has this method loop right here which is essentially tail recursive well future recursive method and it calls loops from themselves and it's infinite infinite recursion it doesn't it doesn't blow up your heap doesn't blow up your stack of words just why it's been running in production for years and so Scala 212 actually have support for that they fix that there's with some really good article really good blog post called acing continuation passing style it actually explains how this stuff works and it says from where when they fix that actually from from beat version of Scala to 12 and 3 I think yeah right so the next really useful thing is interrupts something that allows you to cancel your ear futures basically cancel the chain of your transformations and you can think about that as imagine you design in an asynchronous API you have some promise within your API you will turn it back to the user as a far as a future and the user can signal your back that they're no longer interested I'm the result and you have to I mean you can use it this information to you to your all need so you can you can cancel your calculation or do something like that this is really useful let me show you like imagine we have this promise here the one we want to return from the API to the user and user gets that and do some transformation with that again some I don't know a single bound race right we want to introduce some future chain to actually show that those interrupts are being propagated so user can do like flat map or something like this to anything and so what's interesting here that the new promise isn't state transforming and it actually saves from beach promise it's transforming from and so what happens next we can set an interrupt handler for our original promise within our API and it takes a partial function from troll but we're not really matching on it we'll just say you know don't care just print the stack trace secretaries and the P is now in state interruptible and if we query the original problem when you promise again it is actually transforming from the promise that is interrupted also it's ready to be interrupted and it actually captures the you know the interrupt handler right here as a function 1 so in then user at some point decides that they're no longer interested in the result they can just raise an interrupt on that they just do something like this exception just using some exception to signal it back and we can see the stack tray is being printed and if we created the state of the promise it says interrupted and it's actually capturing the exception that was used to interrupt it and then you promise that user was doing this it actually has the same stay that's interrupted as well so the whole thing the whole chain of transformation is going to be interrupted and the interruption is going to be propagated back to the very root promise the original one and this thing enables lots of finagle futures like very simple one finagle timeouts is just again three lines of code we see we want to wait for response even timeout and if it doesn't doesn't resolve we want to just interrupt this thing and it's up to the finagle the rest of the stack what to do there's that information and it's actually pretty useful because if the request wasn't written to water you can just drop it on the floor you don't have to you know send it you can save some traffic so last one is really interesting locals it's like red locals but for your futures so instead of following the jvm chats they will follow your application flow or program flow which user usually is asynchronous again some example here so let's see we've got this new local it's a type in the utili library and we can query that it has nothing in it right now so it's empty and so the way it works you can reference that I variable from within your callbacks in your transformation chain and you don't have to worry about what treads those callbacks are being to be executed because you will always see to reliably see the same value that you said in the beginning of your transformation just let's let's just model it real quick we have this function run that gives you a union for example let's see we want to do we want to have two promises so we can have some acing boundary to to run them in two different threads and this one again so and now we can actually produce some computation do some transformation let's do flat map we don't care about this we just print the current value in the local itself and then we do again produce a a single bound err here and print the value even that little local value we reference from here and then we want to round those two promises and we want to do that from two different routes so we have this actually machinery in place to introduce this acing boundary make sure the local is being propagated so I have this function which is which is called shift and run it just spin-ups a new tread and satisfy your promise there's not there's no magic to it shift shift and run and this is the P and then shift and round we want to run them sort of in parallel and this is queue this is it nice so if you'd run it they've got two nouns because there is no walk all set to it and the way we do that we say let temp and then we have the context and we're on our competition got to chance back even though we executed those two different call backs from two different tracks we still see the same value so and it's actually a pretty useful thing too and we use it all the places in finagle the most user facing usage code is something called user user context as you can think about this this way when the request comes in and gets a fresh context assigned and we'll follow that request for the life cycle of the request and you can shop some style that in like traits that you request that you'd let deadlines anything you want there and you will always reliably would be able to query that from within any transformations tab in your application and what so what actually really awesome about context is that they can actually be civilized and be propagated over the wire for you so imagine you can set the client ID on your client and then send the request as your local just your local thing the same as show you here and then send your request and there and that value will be serialized for you this relies on the server and you'll get that as your local and you can use it again just the same way you use your locals basically it's really powerful concept yeah I think this is actually pretty much it and again to the best of my knowledge there's no such thing in Scala futures there's something called local variables and Scala futures from 2014 I think you can model that to some extent but I'm not sure how workable is that solution so that's pretty much it it's pretty much what I wanted to tell you the last thing I want to just mention real quick is I it is unfortunate I guess and inconvenience that finagle and the rest of the triggers stack is using its own future implementation but I hope that I managed to give you some insight into how important it is to have this ergonomic built in into your core abstraction core primitive and how it helps you to express really complicated matters like time out retries context locals and stuff like this in really natural and easy to wait and Twitter features helped us to do that this finagle they helped us to build finagle in the first place and they helped us to evolve it into a very unique technology it is today thank you much I think we have some time for questions if you have any yes [Music] another history here but was there any involvement with the Twitter hook and sip 14 in the wages of Scala futures and how comes Scala didn't implement the yeah some people were involved you can go into C 14 and find out who exactly was everything eros Myers but yeah that's a good question I think those like locals interrupts and stuff like that deal made it to the original proposal because they probably were too complicated to implement in the first place we had an example we had a usage for them at that time but Scala futures didn't exist they didn't have a usage for them they maybe didn't decided to support that even root compression something I showed you like infinite future recursion it wasn't actually thin until 212 they've logged an rate off evident question regarding tracing their future so one of the problems that I've faced while deep bass inference applications is you realize which code path the future actually took there was a blog post about something called tracing future yeah so yeah that's a good question so the idea is it is really hard to it's really hard to debug right and we aware of that we've been trying to make it better and better and the last thing we did was to support something called async stack traces for IntelliJ IntelliJ has this basically API to describe how to follow the asynchronous stack trace and we did submit pad support for that to trade futures so if you use IntelliJ to debug your tour features application you should be able to glue your async stack traces into sort of like a local sex race you can figure out what's going on there it's it's actually pretty helpful I think we have it enabled by default for developers is there something that could be enabled on compiled time that actually a rat time agent or just like a drop-in replacement whereby let's say it might be expensive to use that but if people want to use it for their critical code paths then it could be justified yeah right I think there's been some experiments with that I think some people actually try to add sort of like compile time support for that and it didn't work by the way the overhead reasons because you can imagine how happily to Erie futures into promise is used with into your data centers and just if you add one little thing into every promise you'll probably see that on your charts anything else that's it thanks so much [Applause]