FS2 Internals: Performance

scale.bythebay.io: Michael Pilquist, FS2 Internals: Performance

Recording: scale.bythebay.io: Michael Pilquist, FS2 Internals: Performance

you okay so thanks for coming today I'm Mike Pihl Quist the current co-author of functional streams for scholar I work at Comcast where we do a bunch of cool Scala and FP stuff and do a bunch stuff with video today I want to talk about the FS to 0.10 release and I want to do something a little bit different normally when I give FS to talks I do a lot of evangelization of a library and show usage of it today I want to do that but I also want to show some of the tricks we use internally to make it really fast if you're the abstract you saw that I claimed we just released Epis 2-0 at 10:00 unfortunately how to write that abstract back in the summer and it was a little too optimistic so you're ten zero milestone eight is available right now we have a big ecosystem built on that milestone and the 0 10 0 final release is coming really really soon hopefully before the end of the year I'm depending on some other ecosystem and stuff there's three themes behind the 0 10 0 release I wanted to simplify the API a bit support the ecosystem improve performance and so all the things I'll show you today will be sort of characterized by one of these you know themes for folks that worked with scholars estream the transition from scalzi stream 0 8 FS to 0.9 was a massive transition right as a complete rewrite of the library from 0 9 to 0 10 we had like 14 months of production use all sorts different companies using it all sorts of different open source projects using it and we're able to make some very small targeted changes to the surface API the library you know not a full rewrite by name stretch the imagination but sort of sand off the rough edges so let's talk about ecosystem FS - like things like cats tends to be like towards the bottom of dependency graphs right it's this infrastructure style dependency where you need a bunch of other projects to build on top of it to sort of make it useful for your business problems this is a snapshot from the readme for FS 2 projects that are currently integrating with it in some way you can see there's a bunch of type level projects up here sir she's got some if it's to integration do be HTTP for s I think as of right now almost all of these projects have stable versions published for the 0 10 0 milestone 8 release so don't be afraid of that milestone number I have milestone 8 in production systems you know it's really solid but we need to wait for all of our dependencies to publish final versions before FS 2 can be published by the way if you know of any other projects that are on this list please open a pull request love to do it you got one awesome I definitely wanna update this we do have bindings for a lot of things so you know maybe you're using akka HTTP but you still want to do FS to style streaming for some reason there's a project here that you know links the FS to API to the akka streams API and there's a bunch of other interrupts projects including reactive streams okay I want to start with a sample problem to sort of give a feel for what it's like to write things in a Streamy fashion so I figured you know we could do sort of first principle stuff but I wanted to show something that's more your more unique to the FS to API things you don't necessarily see with other streaming libraries so I had this example this is more or less something that you might see in a an app doing like Kafka integration or something like that so we have this pole value pole is of type IO message it's a program that when evaluated is gonna give us a message from let's say like a Kafka topic right and we want to create a stream and the stream the general idea behind it is we want to go and pull Kafka every so often you know maybe once an hour and in this particular example if we fail like if we go to pole for a message we get some type of runtime error we want to do a retry and we want the retried delay between attempts to have some exponential back-off right but if we succeed then sort of store off the message we got and then any point anyone wants to see what the latest retrieve message is they should be able to pull on an element from the stream and sort of materialize that latest retrieve message okay so we're like this continuous stream of always the latest or Treat message and you want to do this in a way that we can prove resource safety and overall composed this program into a bigger expression without a violating referential transparency so in order to do that the first thing you'll see is the use of the scheduler type so scheduler we're going to say in the IO effect allocate a thread pool of two threads for running tasks asynchronously and the type that I get back from the scheduler on apply' method is a stream of i/o comma scheduler it's a stream because FS 2 makes no assumptions on the effect type that has any type of resource allocation or resource safety built-in F is two builds resource safety on top of that you know that the effect types and so in this particular case we get a singleton stream it's sort of an odd idea right if we're just allocating a scheduler why do we get a stream of schedulers but the idea is that we get a singleton stream back we can flat map into that stream and then generate a program that operates with the stream such that when that inner stream that inner program it finishes evaluation then we know our scheduler is no longer needed and we can shut it down gracefully no matter how the program might terminate right and we'll see this pattern a lot in FS 2 all resource safety is built into the stream algebra so anytime you want to allocate a resource and then subsequently free it when we're done with it it's going to be in a singleton stream structure all right once we have a scheduler we might do something like this here we're using the retry method on the scheduler object we say go up and run that pull program run the pool program but if it fails retry it initially at 1 second delay and then each subsequent retry you know double the delay ok again the type here is an IO message our stream of IO message and the reason it's in the stream again is that we've allocated a resource right we've registered some timer with the scheduler subsystem such that when this retry pole stream is finished executing one and unregister that where you want to unregister any pending retries we then might do something like this to say well let's run the retry pro of retry poll program and when that finishes we're gonna pin to it asleep so here we're using the scheduler dot sleep operation again we're saying sleep in the Iowa effect for one hour so this doesn't like block a thread or anything right this is like a semantic block a semantic sleep but in one hour you know that's that stream will finish its effect right it finished its effective calculation and then emit nothing so the actual type of the expression on the right hand side of the append is a stream of i/o comma nothing and if I have a stream of an i/o message and I append to it a stream of i/o nothing I end up with a stream of i/o message that sort of is you know pulling an individual message and then you know running the the second stream Jeff sports effect of waiting okay then we repeat the overall stream and we do that an infinite number of times okay just keep one running and then finally we get to this async package so fs2 has an async package that's got a ton of different concurrent data structures and combinators built on top of them here we're using the hold option data structure or Combinator this this Combinator is sort of interesting it takes a stream as a parameter and when you know a never returns us a stream back and when the return stream is pulled on what it's going to do is run the parameter in the background right go off and concurrently start executing repeat pull and for each value that's pulled out of that stream it's going to be written into a memory cell like a mutable memory cell that memory cell is wrapped in a data structure called a signal which lets us you know register and and and look at the way in which it gets set in memory and the net results that we get back from hold option again it's another singleton stream it's a singleton stream of a signal but then we can use to decide how we might want to read from the signal in this case we call that continuous method on the far right which basically says anytime someone wants to ask for a value from the signal just continuously give them whatever the current value is okay it's hold option because if someone pulls from the signal before we've successfully pulled a single element from that concurrently running stream in the background we get a none and otherwise we start getting sums right and if then the overall program you know you can see stream type stream stream map io option message that overall program is um only run for like the first three elements and then they terminate then all of these resources we've allocated are all sort of unwound in reverse order and shut down okay so this is sort of the idea behind the fs2 API when all this resource allocation and resource management I'm in a way that preserves functional programming the other thing to point out is that nothing here does anything if you were to look at this latest message you have to actually compile it down to an i/o program first via one of our run methods and that would give you in this particular case we give you like an i/o of option message perhaps or a bunch of other different options and then eventually you have to go and evaluate that i/o program at the edge of your program it's actually you know cause it to do do stuff so when I say like repeat pol was off running the background withhold option I really mean conceptually when the overall program is evaluated okay so if you've used FS 20.9 you you notice that I'm using the i/o type here and this is the cat's effect IO type FS to 0.9 we didn't want a dependency on either cats or Scalzi so we imported the scalzi concurrent tasks data type and sort of cut off some of the rough edges some of the unsafe operations that were there you know I was talking to Danielle Spiewak a few well about six months ago and one of the things we talked about was pulling this task datatype and sort of standardizing on an i/o type he had some ideas on how to make make tasks a lot better and so we ended up with a cats effect project and fs2 moved wholesale to it right so we actually deleted the FS to task type completely out of the library all of our examples now use the i/o type but this is really important I want to make make sure this is clear fs2 does not care what IO type you use right FS to everything in the library is a parametric in in the effect type as long as you can bring type class instances for the type class I mean for the effect type that you want to use you know you're free to do so and so for example monix 3 if you want to use the monix 3 task data type it has all the type class instances necessary to do everything we just did in that last example right right out of the box all right in 0.9 we we didn't want this dependency on cats or Scalzi right and so we had this minimal type class hierarchy so this is sort of like a library design dilemma here we didn't want to produce a third functional programming library but we also needed the abstractions that let us talk about building streams that were parametric over and effective and so we we did this we had like toward the big four right like the 99% of functional programming up here in the in the top left corner and then we had these four others in the bottom right and we really wanted these to be minimal we only one of the things that were essential to represent the programs we were trying to represent so we didn't have things like the EQ type class we didn't have things like semi group and monoid you know that that became kind of obnoxious as we were trying to write the library but becomes even more nauseous for users right to go and use this with an individual library you needed some type of shims library and a bunch of things that we could have done and a sort of a more principled a principled way we were limited because we didn't want to you know build this library up to be huge so in 0.10 fs2 you know depends directly on cats we we use the cats type classes for these sort of top five now Mona terror is in cats core and then the bottom four come from the cats effect project if you study like the bottom four here versus the bottom four in the previous slide there sort of like rearranged and like if you actually look at the operations they're not quite the same thing but again you know as part of the work that when it's a cat's effect daniel back did a bunch of work taking feedback from like me and alexandra from onyx and others and sort of shift it around the functionality and this is the type class hierarchy we ended up with one of the things that's kind of interesting here is that in the 0.9 release we needed this ability to do like these memory cells these like mutable memory cells that we can allocate in an referentially transparent way so you sort of like these em bars right we call them wraps in 0.9 we had a type class that talked about what it meant to create an M bar and 0.10 we do that as like a library function off of any old effect so the the important thing about that is that the ref implementation that you would need requires a bunch of infrastructure right it requires a bunch of structure related to like the way in which concurrent operations execute it's not trivial to implement that 0i9 type class I'm here we don't need to write so we don't need like you know demonics authors to write a best to compatible ref implementation we can derive it directly from their effect instance this is at least the second time this picture has been up on a slide this conference right this is Rob Norris its infographic and this is just to say that like there's a lot of good stuff between all of the big type classes right we saw Rob talked about commutative mono monads earlier there's all this other stuff that by depending on cats you know fs2 can use as appropriate throughout the library but also users benefit quite a bit from okay so here's another sort of I don't know library design type dilemma we published 0.9 I stood in this spot a year ago you know extolling the features of 0.9 and a few months later I was with a colleague and I was doing a code review and I saw this line and I got confused and I thought surely this cannot compile I wrote the eval map function this makes no sense so what's eval map email map is sort of like map right you know you map one like a functor you expect to pass an argument from A to B right and on a stream maybe like when a stream f of a you'd map a function A to B and get back a stream of F of B so email map is a little bit different here we're gonna say give it an A return an f of B so on a stream of F a we take a function from A to F be not a stream of FB right which is an F B and then we're going to sort of evaluate that effect and like lift it back into the stream type okay so it's kind of like flat map kind of like Maps sort of like halfway between so what are all these type parameters task task foo that makes no sense right so looking at the zero to nine source code it turns out in some type of fever dream I implemented the fs2 version of can build from we had this like this loved one type right which I don't know it's so it's trying to say that like I don't that type constructor G find the least upper bound between G and the effect type of the stream you're running it on and compute that is like an output you know love here a blob one this is like variance tricks to make one certain type of program compile and so in zero that's hot or zero at ten I'm really happy to say that email map mostly looks like this right I know that's kind of low in the slide mostly right so this looks like this in 0.10 literally if you go look at the code you'll see this now unfortunately we still have this little bit of variance nonsense to deal with and it basically it involves the fact that like if you try to eval mastermind in fur back like a stream of i/o whatever right so we sort of want to lift pure streams into effect whole streams as an effect or as a you know as a derivative of Ebell mapping and so in order to do that you sort of have to well we either have to convince folks to fix a bunch of things as it relates to like nothing being inferred for type constructors in covariant datatypes or you know we have a little bit of syntax trickery to do so anyway but the cool thing is if you look at eval map this literally is the way it's written that just happens to be another email map a little bit further in the same file that looks a little bit differently okay so again library design lesson learned here all these things have good justifications by the way but when you kind of step away you can see maybe we should do it differently so here is an example where we gave you there's just like meaningless choice right this is silly there's a bunch of different ways to do things so filtering right we wanted to represent all of our really pure stream transformations as like standalone functions that are like universally quantified in like the effect type and and whatever other inputting output types they have so in order to write that in Scala we need to do that as like functions and an object or methods and an objects we had this pipe object with methods like filter filter would be defined like completely you know you know with no constraint on like the the effect type but then that's sort of them not just to work with we don't wanna have to like apply these pipe hello all over the place so then we'd have things like source dot filter to make it worse we had things like undone terminate which said like given a stream of F and an option of some type X turn it into a stream of F and X but when you hit a non stop right you're done well because the input type there's an option like we didn't have syntax for that for some reason right and it's like why would again as a user why would you have to know like why hate to care and then other weird things like this concurrent object was an object that sat in the top-level package it had one function and it called join okay I don't know so in 0.10 these all which you know all would what you'd expect they're just methods on stream okay so we made something simpler so I'm going to make something much more complex now so the sort of the trick of performance in any type of stream library is to move function or move values through the stream pipeline in like batches right it's the easiest way to get performance through any type of stream library and fs2 is no different in 0.8 and prior the collection type we use to move elements around was vector and so when we started using it in production start looking at the performance of that library we're doing silly things like spending 20 percent CPU just creating singleton element vectors right because we're creating these big 32 element arrays for all these single teams so in zero 9 we introduced this chunk type so the chunk type is a strict collection it's finite it has a size and it has you know o of 1 index based access right and then we get specialized implementations of chunk for things like Singleton's for empty chunks for chunks backed by arrays chunks back to my vectors including chunks backed by every primitive array type right so you can move like big unboxed arrays of bytes through the stream pipeline without paying at per element cost and this works really well but like I said I wanted to make it more complicated so we added this segment type so we introduced a super type called segment and this is really a stream library in a stream library so this one's a pure stream you can think of it it can output 0 or more values of a type-o and then it finishes with exactly one value of a type R right so it's got output values and it's got a result value and this unconscious a good way to think about it like given a segment one of the things you can do to it is sort of you know D structure it like say give me the next give me the next bit of data in this segment and you're either going to hit the end in which case you get the result type our back right a left or your destructuring like a head and tail in which case you get a head represented as a chunk right as like a finite set of elements and then another segment that you have to process so why do we do this and the key insight is actually one we even saw in the keynote this morning in terms of distributed computing right the key insight is that there's a whole bunch of work you do in between evaluating your effects and the work that we do between those effects we want to fuse together and fuse them in a way such that we're not doing intermediate value allocation right we knew this in 0.9 Endura at 9:00 we supported map fusion so you could map over chunks and we wouldn't create like all the intermediate chunks as we executed you know the stream pipeline but we only did it from map and it was specialized in the stream interpreter so here what we have with segments is that we can views arbitrary operations as many as you'd like we have full stack safety we can do flat map we can do zip width so if you look at the literature around this stuff like flat map and zip with they're sort of the two like test cases that differentiate like fusion approaches and the way we do it is through this technique called staging and the general idea is that the staging technique we use we we want to take advantage of all of the intrinsic some of the JVM and we don't want to like build a code generator we don't want to use macros we don't want to use LMS we want to do this in a way that's relatively easy to write ok so the general idea behind staging is that given a segment we're gonna do things like map map filter flat map and we're gonna get back now another segment right and now when we want to evaluate that segment and actually look at the output values and look at that result type we're going to compile the segment down to a machine like an imperative machine we represent these machines is something called steps and we're gonna like run those steps over and over and over and over until we hit our termination condition typically that we've hit the result type right that you know that our type and just sit there and like a tight loop and so I'm gonna show you what this looks like so here's segments now I said this is this type is a stack safe so it turns out it's kind of hard to do that so I want to delete some stuff right away because it's sort of a noxious to talk about stack safety so we got like these depth and defer parameters and sort of obvious gates what this thing's about we're also returning this eval type over here that also obvious gates it we'd have to worry about stack safety would basically look like this what we have here is the stage operational in an abstract class called segment and it takes three callbacks right this is this is these are mutable callbacks right because this is like compiling us down to this imperative machine we have a callback call dammit that takes a single output element oh right to unit one called emits that takes a chunk of o2 unit final callback called done saying here's your your final result type r2 unit and what you get back had I deleted that right is a step of our the step is the machine the imperative thing okay let's look at something like two lists right two lists what we're gonna do is say forget about the result type we don't care about it we just want to run the segment and convert all the elements to a list so here you know we kind of jump around a bit but we allocate like a big mutable list buffer and say for each chunk see in this segment you know go through and add all the elements of that chunk into the into the segment and if we look at for each chunk and you know bounce around a bit we eventually get to code like this you know we create some trampoline to handle some of the stack safety we stage there's our call to stage we build the Machine up and then we basically sit here in a loop forever and just say while true run the machine like this is literally you know executing that that machine this particular evaluator actually uses non-local flow control through this done exception so I apologize for showing that but nonetheless we can do things really fast with this approach let me give you a couple other examples real quick let's look at map so map right function Oh - OH - and we're going to get back a segment of o2 comma R to implement map what we're gonna do is return a subtype of segment the output type can be o2 and we're gonna have to implement that stage operation so again just forget the depth into first stuff but we get a bunch of callbacks again we get three callbacks now the the output type is now o2 and so map can be implemented in terms of like staging the outer segment right so when we're stage we're in stage the outer segment but when we do that with the pass callbacks to the outer segment and so if you sort of follow the types through here it ends up being pretty obvious how to implement it like you that you know the types of some guides you to the right implementation in this case in effect we do for the Oh callback for the emit a single value callback we're just going to apply the OH value we get with the function at or to the function f and then emit it through our emit callback right if that makes sense here's how fusion works here's how we avoid the intermediate allocation this second one we say if someone emits a chunk of OHS rather than like mapping that function over that chunk of OHS to turn it into a chunk of Oh twos which would be all this intermediate value no memory allocation instead we're just going to sit there in an imperative loop right rip through each individual index in the in the chunk and use the single emit callback okay and anyway when you when you implement enough of these Combinator's this way I mean including things like flat map and zip with we're able to do this in a way where we confused anything still keep the stack safety and avoid those intermediate allocations now I said chunk is a subtype of segments and so here's an interesting example chunks a subtype to segment but segments all the callbacks you know take chunks as arguments and then when we stage a chunk itself like when we say to chunk as a segment we need to build up a machine that omits the chunk itself right so it sort of feels very circular I was lost somewhere in there so here's how it works when a chunk is staged we create a local mutable emitted state again think of we're building an imperative machine this is like a register in the machine right and we're setting it to false we use the step constructor and want to look at this part right here the highlighted part lines like 29 down we basically just say like when the machine is executed when we get a chance to to run a little bit we're gonna check to see if emitted still false and if it is still false we're gonna set it to true and then emit our self out through our callback okay and then regards to whether we did that or not will indicate we've reached the end we're done so this is great we have all of the type complexity of FP and all of the bugs of imperative programming there's also this really neat thing here and this is also sort of sort of novel you don't see this often with fusion stuff we want the ability to kind of come out of the machine sort of like go from the compiled machine that after it's been executed for a little bit with all with all of these like registers that have been set to various values and kind of lift that back into the pure immutable world and we use this for like like the take operator on segment for example where we can run take through like this big complex segment through all these fused operations and then when we reach the end we can say like well let's come D compile or or or go back to the pure world restoring some state and we can return the remainder of the segment post the take right so you can sort of imagine like take being implemented such that it gives you the equipment we got split okay so I hope you I scared you enough with the internals there or at least enough that you come find me later some other simplifications this is a cool one fs2 internally is nothing more than a big free algebra it's just a big free program right so you can't get access to it the algebra itself is is is inside the library but in zero nine we had two different algebra is we had one for streams and one for pools and it's a very common operation to go from a pool to a stream when when executing stuff and so we had to translate those two algebra is like every single time you you went from that pool to stream we had to do a translation step there's a runtime cost to doing that step what we found is that they didn't really need to be different like this isn't like we didn't make them the same by sort of taking the union of all their operations we sort of sort of moved around the operations a bit and actually found that they are in fact the same so this is really neat that a stream and a polar two different like surface API is for the same underlying construct now there's a catch I'm really full of them tonight it's not a free algebra it's a free C so basically what we've done is taken a free mode add it and added this fail constructor again I apologize at the bottom of the slide here but in effect we can suspend errors in our free construction right because we can suspend errors we have a way to observe them we have this own error operation on error is sort of like flat map but on the left side right so it's a function that says from a throwable that may have been suspended resumed with some other free c structure okay and then we do may be controversial things like catch exceptions that are thrown from the pure functions like if as we're computing the free c in flat map if it throws an exception we catch that and use the fail constructor right so with this which by the way is enough to implement like asynchronous IO type this alone with this we're able to give all the resource safety guarantees of - using this algebra this is it right so if it seems like a pretty big API right there's like hundreds and hundreds of operations the algebra is right now well this slide claims there's eight items that's mostly true there's ten because two more have been added thanks to a bug Rob found about two weeks ago but uh there's really not much to the core internals right there's the output and run which just basically let us get segments out of the stream there's an eval step that lets us evaluate you know f of ours right like some some effect full value we've got like four things for dealing with scopes and resource management making sure things sort of unwind and get freed correctly and then finally they this unconscious Inc thing which is a unique to FS - in that we can do asynchronous streaming operations while preserving the resource safety semantics right no other streaming library does this in a functional way and so on kind of a sync kind of looks like a nasty type but really it's taking a stream as a parameter it says I'm gonna take a stream this happens to be like the UH new type stream right the the wrapper is no longer around it but take a stream s as a parameter and pull the next chunk from it or pull the next segment from it don't do it synchronous though give us back sort of like a future kind of it's an async pool right and then I can do things like race to async pools in a resource safe completely 100% safe way and this is not racing to effects this is racing to async pools which is totally safe alright so in my last couple of minutes I want to give an example of how this performance work materializes I've got four different take implementations and so if you've never used the library often you'll go from a stream object to this pool object it's a lower-level API we can then start literally pulling elements one by one and so a common pattern is you might write functions like your own version of take in terms of this pull API so typically these are recursive functions here this loop internal loop is a recursive function don't worry too much about exactly what this code is doing it's basically just saying decrement n each time through by one and as long as we keep getting one element from the stream keep recursing right as soon as we're done you know shut down the overall stream so this is pretty pretty intuitive once you get used to sort of the recursive API the key line here is then is the else block it says s dot pulled on cons one so on the stream we're gonna start pulling on it and we're gonna uncon a single element okay another way to implement this we can unconcious chunk of elements at a time right so again all stream libraries want to batch their data as they as they data flows through the pipeline so here we can we can sort of go down a layer and say well give us access to the batches give us the chunks so if we unconcious a little more complicated right because now we got to account for like this case where we're taking three elements we got a chunk of ten right so we got a in that case cut up the chunk but more or less you know it's relatively similar and then finally have this even lower level API where now we can access the segment structure of the stream I remember segments are sort of like a bunch of chunks that have been fused together in various ways to various operators again I'm like I'm gonna go through the exact example here but the general idea is we've got a stateful fold right or a stateful scan of the segment structure of our stream right and we're saying for each segment that comes in to do some operation with it this in this case we're using the take operation on the segment and then depending on like whether we reach the end of the segment or whether there you know there were more elements or whatever we changed the overall state in the scan alright so this gonna be a bit of an eye chart so don't worry about the exact numbers the takeaway is that 0 10 is much faster specifically over in the fourth column by the way is the 0.9 native implementation of take so specifically if you do sort of the apples to oranges comparison which is the chunk base take in 0 10 the second column with the 0.9 take you can see it's about 4 to 5 times faster the rows here represent streams with different chunk structures so the the first one is is almost the worst example to use for a the first row the worst example you could possibly use for streaming library because we've got all of our data in one big array so we have already done one big array why are we streaming weeks just using the array API but you know if we did do that you can see that the chunk version is still much faster the element and the segment versions are significantly slower and zero-nine but I think it's the only case where 0 9 is still faster sort of Li the opposite end is this stream unfold 1 the third row up from the bottom this is the one that we all often will create accidentally when working on stream libraries where we've gotten rid of all of our chunkiness all of our batch nests right and now we're pushing individual elements through the pipeline on the right you can see 0.9 was about 40k ops per second segment take is a 162 K so again about 4 times oh I'm sorry I was in the wrong row go up 11.4 k ops per second versus 50 house chaos per second right so there was looking at a bigger even bigger increase for the most common usage pattern of the library so Viktor clang recently posted this he you know performance tuning pro tip optimized the bottleneck and repeat right and I think this is very true but I think there's this corollary that we have to continue to remember when doing performance work and that's that you have to know when to back up and try something completely different right sometimes we get so focused when doing performance fork of just optimizing the bottleneck that we never stop to think well maybe we're solving the wrong problem and that's exactly where we were with 0.9 like I was looking at like again JVM level you know intrinsic surround like how fast we can execute free and Paul said hey why are we even doing that we could we could run through the overall trampoline you know one tenth the amount of time so that's all I have I'm zero ten is much much faster for anyone using it we do have a full migration guide that does go through the full API and introduces the new stuff and we were missed not to mention this paper it's an awesome paper stream fusion to completeness it's an old leg paper it's from this year from 2017 and he implements a streaming API for pure stream not affect all streams but a streaming API in Scala using staging to get the performance of native imperative while loops now he does it with LMS so his his stuff will be as significantly faster than FS two segments so but if you can use LMS you know the staging library you know Oleg stuff can actually generate code that looks like it was hand optimized and anyway special thanks to Paul who's done a lot of this work with me and I think I have a time for one or two questions John's gotta have something cool all right well thanks everybody [Applause]