The Futures are Calling and I Must Choos...

scale.bythebay.io: Chris Phelps, The Futures are Calling and I Must Choose

Recording: scale.bythebay.io: Chris Phelps, The Futures are Calling and I Must Choose

you I'm Chris Phelps I'm from Boulder Colorado work for a company called tendril we do energy analytics and energy intelligence and as we do that we want to try to have reactive systems kind of following the model laid out the reactive manifesto we want systems that are scalable and responsive and do all those sorts of things so we want systems that respond in the kind of time frames that we need for batch traffic or for user traffic we want resiliency so the systems can deal with failure and recover from failure we want to be elastic in order to support those and be able to scale things up and back down so we saved money when we don't need it we can avoid things like batch processes causing outages on our live website and things like that and then there's some kind of message driven stuff that supports all of that so if you're trying to develop that kind of system you want to start looking at asynchronous things and how to manage being asynchronous how do I take some work start working on it and keep from occupying the rest of my resources while I'm waiting for things to happen if I'm building a multi-service thing and I have to go talk to a separate service or talk to some third party to get data if I have to go off to the database to get some data how can I use those resources effectively while I'm waiting for responses to come back from those other services this is the end of three days it's scaled by the bay you've probably seen a bunch of talks that have talked about asynchronous behaviors and asynchronous effects and how to manage effects in your system and we'll talk about futures which are a good mechanism for us to do that asynchronous effect handling so kind of a really brief history what's going on with these future things right like so way back in the day probably 2006 or 2007 we started developing Java this future representation that let us represent a value that has not completed yet but at the time we really couldn't do anything with that we could say is it done yet or let me block and wait for it that's not a whole lot better than just blocking and waiting for the thing in the first place Twitter and akka came along and they started building more sophisticated ways for us to manage that without having to just sit and block on the thing start giving us callbacks when the work is done and that sort of thing and then in 2012 with sip 14 we moved that into Scala standard library and built the first Scala future library so we've still got Twitter stuff on from that time frame and the akka stuff when we had to 14 all kind of deprecated and moved into standard Scala library stuff so Scala standard future we've had around since 210 it's continued to have things happen with it and actually I don't have on this slide but in Java eight we started to get Java completable futures that built more of this stuff back into the Kaurava so in this talk I'll kind of go through Scala futures how to use them twitter futures how they differ to scala futures I'll mention really briefly guava you can say boo now or then talk about completable future will touch on tasks for a little bit and then we'll give you some kind of comparison summaries so let's start with Scala standard future so what makes up a future in Scala basically we've got three states that a future can be in it can be unresolved resolved successfully and it has a value or resolved with failure and it has some sort of throwable associated to it we have an ability to make immediately resolved futures for purpose of chaining together pipelines so futures which are immediately resolved successfully futures that are immediately failed then we have promises I'll talk more about promise on the subsequent slides promise is a way for us to grab the starting point of that computation past someone else the future back from that and then later resolve the promise to resolve their future and then there's a bunch of Combinator's for putting these things together and working with them it's how do you get a future well the trivial answer is you call somebody else's library and it returns a future but that's not very interesting because it just raises the the subsequent question how did they make the future in the regular scholar library we have apply on the future object which will create a future body and put that into an execution context or as I just started to allude we can make a promise so let's start with apply we have to have an execution con execution context that these will run in I'll talk a little more about that on subsequent slide in this case where you're just going to grab the global one then we use future and pass it a body and this calculates slowly is going to happen inside the execution context that's going to yield this future event that I can pass around and do something else with creating one with promise on the other hand I will make a new promise I will return my client the future end of that promise so that's how my client can wait on my result I can go off and do my calculate slowly and when calculate slowly is done I fill in the promise with the successful value and the client receives that as completion on his future so the client can set up his callbacks and do all that kinds of thing and he gets those callbacks called when I resolve the promise and so the way this works we've got a producer in a consumer producer is going to build the new promised return that future to the consumer consumer is gonna register callbacks on that at some point the producer completes the promise and the callbacks are executed on the consumer so callbacks so in this kit example here I've called the calculate method and calculates return me a future I'm going to use on complete to give a partial function that partial function and when it receives a success can do something with the result when it receives a failure can do something with the throwable that's inside that failure so that's a single partial function that's given to on complete that handles both sides of those but what if I want to build sub parts of that I just want to handle the success case I just want to handle the failure case so in 211 we have we have actually an on complete so this slides not completely up to date have an on complete which takes a partial function and just operates on the success this is only called in the case when the future results successfully and a non failure that's called when the future fails and is a partial function that looks at the at the exception so in this case I haven't done anything particularly interesting with res or with T but that's a full partial function we can do as much as we want maybe in our T case we look at different kinds of failure and respond differently in those cases maybe res is something non non trivial and we're doing some kind of matching or some kind of more to do work in there in 212 both on complete and on failure are deprecated they're still available but the preference in this case is to use for each I guess the one thing I don't display here is what's the what's the signatures of these right so the on complete and the on failure were both partial functions yielding unit and in this case we make that kind of more explicit with for each and say this is a thing that is called and does side-effects it's not returning a value it's a unit and then we have this failed Combinator which lets us take the failed one turn it from a future of a to a future of throwable and that subsequent future is then resolved successfully with the failure from the first future and that's why it's failed dot for each but rather than doing those things that are just side-effect II callbacks when the future succeeds or fails instead we prefer to build chains on top of these and chain futures together so that they can do other things and the main mechanism we have to do that is map and or maybe the first mechanism we have to do that is map and flatmap we see these on many many types in the Scala standard library and they all have basically the same signature so map is parameterize din s this is on a future which is itself parameterize din T so then we have a function that takes a T and returns an S you pass that function to map when the callback sorry when the future resolves the when it resolves successfully this function is called passed the value within that future it returns the s but have built a pipeline here right so what comes back from that is a future of s so you pass me this function now and I say I'll give you now a future value of what your function returns to me flat map lets us take that up one more step and says the method that you're gonna give me itself produces a future so the map case this is something that's going to happen right now the flat map case this is something that is going to yield another future and I want to chain those two futures together so when I get a call back from the first service I make a call off to the second service and those two future contacts are chained together and now ia or client can set up callbacks on to that so let's look at an example of that so I have a calculate method that makes this first future I have a simple method that takes an int n returns an int called scale and I'm going to hook that up as callback on the initial future using map so when my future resolves the value that's in that future is going to be passed to scale scales going to calculate it and that's the future int that I get out of all of this so scaled itself is a future recalculate is a is one that actually that instead adds another layer of future so when so we flat map this when the first one succeeds we'll call the second one recalculate we'll create a new future we'll chain that together recalculated is now a future that I can set up callbacks on or do something else with from there and once we have map or flat map then we have some standard Scala syntactic sugar called for comprehensions and these let us chain these kinds of operate these kinds of classes that have maps and flat maps together and just something that looks more iterative right so we could have done this with a bunch of nested for nested flat maps instead we'll use the for comprehension to chain these all together so I get I calculate I have a future I recalculate I have another future and then I can scale the final result and this turns into underneath a set of calls to flat map flatmap map to yield these all together so these are in equivalent to calling individual calls to map and flatmap and I I don't have a slide that shows what that D sugar is but it's a very standard thing you can look it up so let's talk about some other Combinator's that you use to start putting these things together so a couple that I've displayed here fold left is a way for us to take a list of futures and combine their results together in some way so whereas the all of our previous examples were chaining one after another this is I've got several results and I want to do something to combine all those results together into one result where these are typically results of the same type I make three different calls out to my user service and then I want to combine them together to have one final user object or something like that first completed of is a way for us to race several futures and we'll get back the first one that finishes so maybe I'm making a call to three different nodes and I want to take the first one that comes back and sequence and I put sequence on this slide because it's the simpler signature but this slightly more powerful version is called traverse this lets us take a list of futures actually a traversable of future so strictly more traversable things than just lists but it's reversible of futures and turn them into a future of traversable so I have my maybe I go off to to a user service and I make three different calls to the user service to get three different users and I want to turn that in at the end to one big list of users I've gone from a list of future users to a future of list of users by using sequence or Traverse so let's talk about execution model so these things have to finally run somewhere right and the way this works for scala standard future is with this execution context i showed it at the beginning that we were grabbing the global one we can also pass that explicitly and pick a different one so what happens with this body is the body gets submitted as a runnable that's put on to that execution context and it's up to the execution context to decide how do i allocate that to threads all right so the execution context maybe thread pool oriented or something like that and it's taking that callable and deciding where and when to run that callable key here is that every one of our bodies goes back to the execution context and the execution context and potentially puts it on its own thread or does whatever so in the case of the global execution context that I showed before that I was grabbing implicitly that's a fork/join pool is what the standard one is implemented as and everything that we submit as a body in a map or a flat map becomes its own callable gets its own thread so but we can also write our own executor or execution service to do different things and that's some of the tricks that we play later on to look at some of these other execution models that other frameworks do why might you choose to use the implicit or use something else on control over pool size if you're doing a lot of long blocking operations there there are other pools that are better for that if you are managing your own set of pools and maybe you have a general pool that you want to do a lot of things with and then you've got a special pool that you use just to do just to frogmen to keep the things you can send just this result to the Frog Nicator pool let's talk about Twitter so there's some differences between Twitter features and Scala futures and any of you messed with these you know have run into those very quickly so the combinators the execution model and cancellation are the things that we're going to talk about here I was told just a couple days ago that locals are the best thing about Twitter phew and I'd never heard that one before the documentation for them isn't super great so I'm not going to talk about them now but there's another difference so your bonus so so Twitter futures have most of the same the same kind of combinators we've still got filters maps flat maps flattens for each's and with futures sorry with filters I didn't talk about a lot of these in the Scala standard one but many of these are there in the Scala standard one as well filter right lets us lets us return or not return the result based on whether the predicate succeeds flat map we talked about before flatten is for when we have a future of future of something to turn that into a single future by chaining the two futures together for each we talked about before that runs a side-effect II method when the results their map we talked about before that return that runs an on-site effective method when the thing is there and with filter is actually part of the underlying mechanism for how the for comprehension guards work but again takes the result and applies a predicate operation to it but kind of more interesting or what are the things that Twitter futures add that Scala standard futures didn't have so a couple of these at the top or by in within these take some sort of timer and some sort of time so by lets us say resolve this by this time within takes a duration let's resolve this within some amount of time so you typically use that for some sort of scheduling kind of thing this is not really a time out sort of thing we have parallels and selects and wins and wild dews all of these are about using futures to control other futures so parallel takes and runs the same operation end times in parallel and you see it gives us back a sequence of futures select lets us take a sequence of futures and grab the first one that comes back and the list of the rest of them that are there so it's it's like racing without discarding all the rest of the things that fail the race when is so when you see is is eager it's not taking a lazily evaluated predicate so when this case is true run this future and weíll do is reevaluating so this is while this predicate holds keep evaluating the future and that says future of unit but that's surprising to me if that's really doing iteration but the theme of these right the theme of the these latter ones is is observing and choreographing other futures using futures to control the behavior of other futures and control whether we proceed with computations many of the standard Combinator's that we have in scala standard library are also there in twitter but have different names so is completed versus is done value in standard library versus poll in the Twitter future recover in the standard library versus handle and Twitter recover with instead of rescue zip instead of join zip with instead of join with but I lied to you a little bit a bunch of these in the Scala standard library take the execution context but in Twitter they don't so what's up with that so the Twitter execution model is different from the Scala execution model because it's based on executing eagerly in the thread that resolves so we end up with this pipeline that tries to stay in the same thread you can fork to another pool but you're going to try to stay in the same pool so when you when the when the first future resolves the thread that that future is running in runs the callback and if you set up a callback in the futures already reserved already resolved you'll execute that callback body immediately in your thread so there's actually an interesting thing that I discovered when I was first putting this talk together and I was really surprised about why the thing failed in a certain way and it turns out that I was running synchronously to me because my the future that I was setting up and trying to wait on was already resolved just the way that I was building my test case I had a very quickly executing future I immediately tried to add a thing to it and instead of that happening off in the future in the thread of the the work that I was trying to do I did it right away in me and I ended up with in my thread and I ended up with context for my thread not context from the running thread so future dot apply runs the body immediately in my calling thread so if I want to run in a new future how do I do that that's with an explicit future pool so I've built here a an unbounded pool if I immediately just do future and calculate this is gonna run right away in me and you know if this calculates expensive I'm potentially blocking myself right versus if I do on the pools apply then this execute it gets submitted to that that future pool I get a future back and then I can run along and do my thing another big reason that you might use Twitter features is cancellation but cancellation is not as permanent as maybe you'd think so from the promised side you can interrupt by using rays that you pass a exception to the other thing is your raise it the future is not resolved but it's marked as interrupted and then inside the running calculation you can check for interruption I'm gonna speed up just a little bit there's also a raise within that can add a timeout and will automatically raise your future with an exception when it's when the timeout expires so let's look at that I have an unbounded pool I make a promise inside my inside my body I'm gonna do the expensive calculation and then I'm gonna check and see if I'm interrupted and if I'm interrupted then I'm going to finish the future if not then I'm going to perform the next calculation so I need to check both between expensive parts whether or not I'm interrupted and make a decision whether or not to continue and then from the outside somewhere where we've got the original promise we can fail that with a with I give up that causes the future to be interrupted and allows the body to check that by jek ssin if you need to operate in both Scala futures and Twitter futures by ejection library lets you convert between one or the other so you pull in some imports in this case I've renamed them Scala future and Twitter future I've pulled in the as method which gets me some some syntax and the by ejections themselves to get all the conversions and then I can do response which is a Twitter future dot add Scala future to change it between and what this is doing internally is setting up callbacks to make this happen that's not just a typecast but an actual wiring up of callbacks so that these two things work I'm gonna say very very little about guava guava is a great library and it's time it's really painful to use from Scala in particular listenable futures are very painful to use from Scala they don't chain the syntax is just not comfortable from Scala if you have to do anything with them you can use some of some third-party things to buy check those together so there's this thing called guilt foundation guilt Foundation classes that provide a lot of these my ejection to convert between the two in our case at tendril we have a lot of legacy code that was built around Java and listenable futures so we really do care a lot of times about converting back and forth between those but if you don't need to do that I would at this point avoid it as Scala developers so let's talk about Java 8 so in Java 8 we got completable future it was trying to take more of the sort of future ideas back to the Java ecosystem that gives us very similar rough shape to what we have in Scala we've got a future with callbacks we've got some sort of pipelining sort of behavior with promises and futures in the completable futures case the same object is both the promise and the future it's both ends of that pipe we could create as a promise we can use supply async or run async to create runnable bodies that get submitted to the executors so an example of that so I have a completable future I'm going to apply async this scale transform then apply async is similar to map but it's going to happen in another in another call to the executor instead in the same thread at the end we have a handler the handler here this is a very Java like sort of signature even though I'm showing this from Scala so we get past both the result and the throwable and if this future completed successfully we have a non null result and a null throwable and if this failed we have a null result in a non null failure so we have to take this we have to say if R is null we do the happy thing if otherwise if T is null and we do the exception thing so we don't have to four methods that are called in those two cases we just have one method and then this behaves this completable future behaves like a promise as well so at some point later I can complete that and then anybody who's got a handler registered will have it fired so the execution model is sort of a mix between the Scala execution model and the Java execute the Twitter execution model so we have two different methods to indicate whether we're sync meaning that we execute the body immediately in our thread or async meaning we submit the the body back to the execution context so we've got pairs of many many of the methods apply apply async handle handle async regular version when the future completes we're going to execute it immediately in our thread the async version when the future completes we're going to hit submit it back to the execution contacts and the execution context potentially is allocating a new thread to do that default executors we get is a common fork/join pool based on the number of cores and what that type of pool does is typically makes a new thread for every body there's some configuration there that prevents that from happening but typically it's going to create any thread comparing the Combinator's between the two we have variants to deal with runnable 's and functions suppliers and consumers we don't have a recover with so we can't take a failed future and replace it with a new future operation that's going to resolve it we can just recover exceptionally and handle the problem and do something with the problem now so I didn't really cover this but in the scala version we have recover with and you say recover this failed future with another future that's meant to replace it I'm not going to mention any of the rest of that let's talk about tasks so for purposes of this we're gonna think about tasks as lazy futures futures that don't immediately start to execute so all these other ones that we just talked about they're starting execution of their body potentially right when we create them right in the Twitter case we immediately started to do that on the collar thread in the Java case we submitted it to the execution context and the execution context is potentially creating the new thread as soon as immediately to go start doing that work but in the task case we're not going to start that computation running until we actually tell it to run so we can explicitly run it with actually one second the a few of them that I'm talking about in general here scholars Ian's monix both have roughly similar shapes and roughly similar sorts of semantics fs2 has its own implementation of tasks so that it uses internally and I believe they're migrating that to Katz's effects in newest versions john de gos gave a good talk a couple days ago about scala 8io and then there's also work going on in the cats community to make a slightly more general effect for asynchronous deferred computation so Scala 8 IO and cats effect so one of the the upshots to not evaluating immediately is that we could potentially evaluate the same future multiple times and the reason why that's beneficial for us in tasks is because we have the option of whether or not to memorize meaning save the result the first time that it's available so we can build a task build this whole pipeline around the task and we can choose to not memorize it and just run it now and then run it again in a couple minutes and then run it again in a couple minutes so we've set up a pipeline that we've reused again and again or we can choose to explicitly memo eyes that say run at this time store the result essentially cache it for the rest of the life cycle of the of the objects execution contexts execution models are typically trampoline and I'll talk about that a tiny bit more and there's some cancellation to various degrees in all these these models so I'm mostly going to focus on monix here so in monix we can create tasks using apply we can run that task and give it a handler at that time and that handler just like we've seen before is going to execute when the things available if we want to explicitly run and memorize the result we task that memo lies if we want to cancel when we called our run async we actually got a cancelable future back and then we can cancel that so let's talk about the execution model execution model for these is something called a trampoline it's almost always controlled by some kind of scheduler in the monix case the scheduler has a few different modes that it can operate in the default one is batched execution and what that's going to do is try to keep running on the same thread as long as possible and then periodically force forking to a new a new thread so after you reach a number of tasks on that queue fork to another thread to do more there's always a sync which does everything in its own thread and there's synchronous which tries to do everything on the same thread and so what happens is we submit these units of work to the scheduler we're going to execute we're going to execute those on the same thread within the executor whenever we hit a failure case and we need to clean up or we hit an async boundary then the scheduler is going to fork to a new thread but we can also explicitly fork as we go through that so in our pipeline we can say I want this next piece to happen on its own thread fork or we can use this acing boundary thing to cause an opportunity to fork so we I mentioned up at the top a sink boundary is forced after a certain number of of pieces of work a sink foundry lets us force that boundary at the place where we where we want to put it the purpose to that is we want to either fork to any thread or we want to offer the ability for the system to forcus to a new thread or we want to offer the ability for the system to cancel our thread at that point so this is very much like the interruption status that I showed before this is sort of being cooperative and saying here's a point in my workload that that I could pause we can also use async boundary to switch to different schedulers so in this example I've got a task which is always forked to a new scheduler to this the i/o scheduler I have my initial tasks in source I'm gonna flatmap to switch to the i/o scheduler I'm going to async boundary to switch back to global and then I'm going to set up an on finish and I'm gonna run this whole this whole pipeline so let's look at a at a comparison of at least the future parts that we talked about we've got Scala JA Scala Twitter guava and Java promise like creation in Scala and Twitter comes from promise in Java it's all completable future we can a create async via apply in Scala on the future companion object in Twitter on a pool and Java we can supply async to submit a callable to the execution context execution model Scala runs in an execution call context so everything gets submitted and potentially gets a new thread Twitter the completing thread runs the body Java the pleading threat or the executor runs it depending on the method that we use when we built the pipeline supporting for comprehension Scala and Twitter are both based on map and flatmap so that you can use for comprehensions to build these pipelines Java does not so you have to use the other mechanisms but you can play games with with conversions and extension methods to to add map and flatmap that call to the right Java methods Scala does not support cancellation twitter supports cooperative cancellation Java does provide some cancellation capability Java Scala interrupts the Scala one you can implement it the Twitter one provides both types in its in its API tries to provide a Java friendly API and Java obviously Java interrupts with Java classes is really good but there's some type inference stuff that goes on you have to put more type hints in there than you're used to so what do we recommend I mean the biggest thing that we always say is use whatever your libraries use if you're in a finagle stack it's really painful to convert to Scala futures everywhere just so that you can deal with Scala futures in a bunch of other places if you're in a finagle stack you know maybe try to stay Twitter all the way through and vice-versa if you're in a in a Scala stack maybe it's not worth trying to use the Scala are the Twitter futures unless you you need to so stick with Scala unless you need cancellation or you need the monitoring or or you want to use locals if you need to mix Scala and Twitter definitely use my ejection it's very solid and easy to work with if you need to interrupt with Java prefer completable future stay away from guava it's just too painful to use from Scala without buying you a whole lot and by all means watch this space you know there have been several good talks this week about Scala 8io a bunch of just kind of general buzz around cats effect over the last couple of days and and all these things are moving so take a look at other talks and pay attention what's going on in the space really quickly a few things that you can look at if you want models of asynchronous other than futures reactive reactive streams streams actors and co-routines are all different things you can do that's me I hit my time precisely if you have any questions you can come catch me later thanks [Applause]