Channeling the Inner Complexity

Scale By The Bay 2018: Jakob Odersky, Channeling the Inner Complexity

Recording: Scale By The Bay 2018: Jakob Odersky, Channeling the Inner Complexity

you so welcome everyone before I get started I'd like to run it a quick show of hands here a quick poll so who here uses go on a daily basis all right nice who likes the concurrency model of go everyone who doesn't know should put their hands up okay who uses collimated on a daily basis one day come on okay so today I'm going to talk about scholars basic concurrency mechanisms and how we can leverage those to build a library only a library no language extension that enables us to emulate ghost concurrency model so what that means is having channels and lightweight threads that don't actually run on real physical process that threads to get to that point there's two concepts that are important which is parallelism and concurrency often they're used interchangeably but they actually refer to two different things parallelism is the concept of actually running a part of a program at the same time as running another part of the program and concurrency is building splitting up your program into parts that can then be run in parallel but not necessarily in parallel but you still get the same outcome in regardless of what order you run them so the premise of the talk is that I claim any scaleable programs are going to need a good concurrency model at some point by good what I mean is program that lets us be as efficient as possible use modern hardware so take advantage of parallelism and also reduce the complexity in which we structure the program that's also a very important point to start out there's a the most one of the most basic concurrency abstractions which is the thread so a thread as many people probably know as a sequence of instructions it has a single entry point and it's a traditional way that's being used on operating systems for a while every thread has it stack and Colonel resources so that means it's a fairly expensive resource I mean you can still in modern hardware running your thousands of thread but not millions oh you're gonna get in trouble the other thing is every thread that runs on if the thread is running on a processor gets taken off the processor a new thread gets put in that's also a fairly expensive operation a context switch but then the advantage like I said is threads are actually the unit that can be run on processes it's like the native thing for the operating system to illustrate this example here we're gonna look at the make meme method it's gonna follow us and to many other concurrency models as well this is on a single thread the most straightforward thing you can imagine what is the meme it's an image superimposed by another image which is the text on it so if we to create a meme we need a URL and some witty text or something on it this program here is even someone who doesn't know Scala can very easily reason about it right we get the layer one that's we fetch an image so yeah we just do the network call until we receive the result the second layer of the image is we convert the text to an image and then we superimpose things so very straightforward anyone with any programming background is probably gonna in a reasonable amount of time be able to determine what happens here the unit of concurrency here is the single program you can't you can run a conference of the program in parallel it's like you run the whole program or well you don't there's no change in outcome the problem with running this in not a problem one of the shortcomings of running this as is is that although it's very simple it's very wasteful right fetching a URL is we're sending off a network request then we have nothing to do until we get the response back network requests takes milliseconds if we're you know in a good spot processes in the order of gigahertz so we're just like not doing anything sure you know if your computer's running other programs so if the unit of concurrency you have others than the same level then you're good but usually like if you if you have a server this can become a bottleneck you're not taking as much advantage as you can the other thing then is of course is to get the results you need the result the final result needs the result of two intermediate results but you're only kicking off the second intermediate result once the first one is that the classic example rather the most straightforward improvement would be to just run these two in parallel this is where multi-threading comes in like Scala pseudocode it would look something like this we define the results we kick off two threads run things in parallel we need some synchronization and like glossing over the details because that's actually one of the big difficulties and threads making sure that your program is correct still looks straightforward but so as I mentioned is any if you have multiple threads to do anything useful in the end you need some kind of synchronization some kind of rendezvous because otherwise the results are pretty useless in hardware that's that that's typically implemented in hardware and very low-level instructions but as a result it means that reasoning about your program is much much more complex especially if you need to ensure if you want to ensure that you're not going to run it to deadlocks or any other kinds of issues and then furthermore is that threads even if they are blocked so even if you're not running them because they're waiting on a network call there's no consuming memory we can say again like memories cheap just throw in some more RAM problem though is that we still have the issue of context which is if we are dealing with many threads so what I'd say is well they're a low-level building block but it's hard to use them as efficiently as possible they're central but very hard to use right also they're not available on any platform think of if you're using javascript do you get one thread you don't you can't run many things in parallel so just to make that synchronization complexity a little bit better is often times we structure programs especially large programs around queues it enables us to decouple our logic you know we have producers and consumers I put something on the queue something happens I don't care what happens with it but you can model programs and reason about them fairly nicely with queues this make meme model would look something like this you see it's still very similar to the previous version except we no longer need to deal with synchronization the assumption here is that we built the cute that does synchronisation correctly at put and take so that means if I'm taking an operation there is no operation I'm just blocking so the result of the program still safe so what that gives us is we have a simple logic flow however we're still using exactly the same resources as threads so if we step back in second I think of what does it mean actually a thread is blocked right like what runs a threat well that's the operating system the operating system itself it's just this big reactive state machine that takes input does things like a timer a network card interrupts so what if we can rather than trying to work on the abstraction of a thread we try to take this callback so this mechanism where we react on events if we use that into our programs as well so that's what callbacks are yeah a program with callbacks is typically can be seen as having many entry points so you react really on methods typically how things happen if you don't have many entry points is when you start up you register operations so functions or that's where the name comes back callbacks that should get run when when an event happens typical examples all of JavaScript libuv or any kind of event loop system essentially and in a sense they are as fundamental or sometimes you could say more fundamental than threads because it's just absent reaction now back to our example here you'll notice that we were still doing the same thing but the signature of the method has changed slightly so rather than now returning a result we are accepting an additional parameter which is a which will be called when the result is ready same goes with the fetch URL we assume that this method now rather than returning a result also takes in a handler and you'll notice that the logic is just extremely complex but a simple thing we were doing before all of a sudden we don't have this linear flow anymore to reason about what's happening is we need to know the big picture of everything we can't think about things in isolation anymore it's my Plus also there's an error if we have we can run into a deadlock issue here or return to results if parallelism is greater than 1 here so things like that you need to think about - if we actually ran this this is I think what we would get ok so the advantages now of call backs there are we never needed to allocate any additional memory for a thread we don't need to allocate extra extra stacks we didn't have any threads that were then would happen require a context switch that needed you to take an off the process and put back so callbacks are very efficient they don't have much resource overhead they're available on any platform even if you have a single thread you can have as many callbacks as you want and finally you can run them on many processes as well the disadvantage of those will make things become extremely complex streaming quickly and that's main callback hell are you thought of that so now what if we could rather than register and making callbacks how work purely on side effects are only registering changes what if we could wrap them in a more functional way so now we deal with transformations rather than side-effects all the time keeping the same characteristics so the efficiency but reducing the complexity this is where the Scala futures coming probably seen these things what they do is they wrap a single operation that eventually will return some result at some time you can transform them with maps flat maps all the other goodies you have on collections and when an operation is finally run the future is going to one day complete either with a success or an error if we look back at our in our program this is how it is evolved now so we again we're now returning a result but rather than return the actual result we're wrapping it into a future how we get the two images is again we assume these methods here would return futures to us and then finally the way we compose the result is with a simple for loop and this is essentially just mapping over the for loop in Scala and this it's just a shorthand for running a flat map to flatten out something so program again looks much simpler still I would I would say still not as simple a little bit more complexity than the first single printed one but at least we can distribute this on multiple threads now though you'll notice that I kind of said like oh we assume we have this fetch ERL text to image but at some point futures they don't just magically exist we do need to interact with the environment this is futures fundamentally wrap callbacks at some point of this callback graph that futures are going to build we need to interact with it and this is where promises come in so it's kamek promises can be sources of futures so you can say I create a promise which is this box where I can get a future of it I can do my transformations on the future and then at some point in a callback for example with my environment I can imperative Lee complete the promise and then trigger this whole running of that blueprint that I constructed with the future as an example again this would be in Scala J's assume we want to build a browser within a browser because that's like what you do in JavaScript these days right so we we need some way to get a URL from an input text box to be like our URL box you register here the callback complete the promised return the future and how we actually use this is on the URL that we get which is the future we would map something then gets the URL displays the website and then again to interact with the environment one final callback handler thing though is we just have callbacks at the edges not in between the operation so now then the question though is how is it actually run the future we could say that whoever completes the callback runs the entire graph from though we don't really want to do that otherwise we have these huge chunks of concurrency otherwise we're again limited by the by how we can divide the program so what we do is instead the operation that every futures every transformation on the future will itself become one of these chunks that gets scheduled on something called an execution context so if you've used futures you'll know you for any operation on the future execution context passed as an implicit parameter an example here would be let's say we have some future we flatmap on it we have another future we map on it again in if we were to explicitly write execution context you'll notice we need execution context for every operation on it because these are going to be the cutting points to create these chunks that get put into the execution context execution context is then run on a thread pool thread pool is just a limited set of threads that have a work queue so they will take these chunks run them when they're done call its another callback but this is all hidden from you to to run the next element and this is why you've probably heard that you should never block in futures and this is precisely why because if you do that you're limiting the work if you have a limited set of threads you can just like run out of threads that you can do every other one is waiting if you don't have a limited set we're back to square one what we said were you're not taking advantage of resources efficiently okay so futures you know are nice to use but there's still a little bit of too much complexity in my opinion that is futures are hard to compose well let's take you a little bit more complex example we want to look up a user it's gonna be in a database and it will call anything and it might return a user or not we don't know so result is a future of an optional user we also have another method that for a given user and the set up capabilities can authorize and say like does this user have the authorization or not what we return though is an option of user because we if we do not have the authorization we don't want to reveal that we want it to be transparent to anyone who calls us like if the user exists or not so they can't get any information out of it if we want to combine these two methods so to authorize a user by ID now we ran into something complex like this you need to you need to essentially we're trying to do is like map on the inner user so you need to unroll the whole result set do your transformation and wrap it back up and that can much more complex if you're dealing with these very nested types of course you can like restructure your program around it but it's still just like the it's fundamentally they don't compose well this is where you might have heard monad transformers it can help you with that but I still think you know they help you but it's still an advanced concept someone who's totally new to the language she comes away from an imperative background looks at monad transformers in Rick whoa what is this so they can help but it's still not as simple as having this linear workflow so all right so the two shortcomings of futures were their main composition as I mentioned then the other time other thing is that futures are a one-time result at some point they complete it's not easy to model recurrent events it's not a cube it's just it's gonna happen or not so the solution to the first shortcoming that complexity of composition is can be solved with a library called Scala async what it does is enables you to write code that looks synchronous so back to that first example I showed but actually doesn't doesn't use actually candy compare works with futures and can decompose these into it has its own state machine which which can then run these futures bit by bit but the code that you see looks totally synchronous I get to an example in the same it does this by offering two main constructs which are async and await you see async the way it looks like is you put in some body that returns a result that we actually will get is a future a weight that you can only use inside async you can call it on any future and get back as a result this is again just syntactic sugar right in the end like you can't there's no way you can actually you could do that without blocking its it's a macro that does that rewrites your code it's also an official project from the Scala center that means it's not gonna go away tomorrow I'd highly recommend to check that out it makes your code dramatically simpler if you have many nested future calls this is how the program would look like essentially exactly the same as showed in the first example except that we now wrap things in an async block and every layer gets this weight in here and so super simple logic again operations are kicked off in parallel so we get all the advantages but the logic flow for the programmer looks looks very linear super easy by the way if you're familiar with with futures in general Scala you might have seen the capital await this is not the same thing it's totally different so capital await will block a thread and they say that you should try to avoid using those whereas these are just it's just syntax trigger okay so that solves the first chart coming there's still another shortcoming though which is that futures are a one-time value they complete but then then that's it you need to create in your future if you want a new result so how can we could we work around that ideally what we would want is something like queues and threads again because this is how you model many interdependent interactions problem is as we've seen is traditional threads and queues they block and that's kind of what we've been trying to avoid all along so this is where this project that I've been working on comes into play project is called Eskow which means I can stop because it's like it's working like on a state machine as many stocks in it it's inspired by closures kordon async library which itself is inspired by Scala async so we've come full circle I highly recommend to watch rich hickies presentation on it it's excellent explains all the advantages there's also talked about some implementation in it so that's about the project now what it actually does is it offers some extra constructs there's something called to go which creates like co-routine so would run the block itself it will return the future I'm just showing these here for completeness you're never gonna need those you have the concept of a channel which is like a queue you can put something into a queue into a channel and you can take something out of a channel the definition again would have a future but in the syntax you will never see it it will look blocking plus there's some extra utilities like select which is you can call multiple channels and you'll only get one result one and only one when it returns it represents a form of communicating sequential processes which means that there is a formal if the implementation is correct right I'm not claiming that it is but it would be nice if it is then that means you can reason about about your program you can actually put reason you can prove that it's not going to be blocking and things like that and also since we don't rely on threads the runtime slowly abstracted this can run on Scala jeaious scholar native and of course on the JVM can use as many threads as you like this is how an example would look like we have across we need an implicit execution context because that let's make this whole thing work then we can declare yes we can create a new channel given the type by default channels have no size so that means if you write to it you're just gonna lock I put block in quark marks because again I'm actually blocking the thread if you arrive in that situation your program is going to relinquish the thread thread is going to work on something different and you can read from them and the way you would start these synchronous operations is with the go contract again go is not starting a new thread it's just it's just giving you the the conceptually it looks like it is another example is this example here so we declared two channels and then we start to curl routines where we would write to both channels the order here is not going to be guaranteed so if I run right this first doesn't mean then this is actually gonna happen first it could happen that this thing here happens first again these things depending on your environment can be executed in parallel or sequentially but it's like totally abstract it from you then we can use the Select call where we await one of the and and this is kind of a primitive it so only one will complete you would never get both this is a little bit of a primitive example but the this whole concept applies can be used to model things like sockets web servers in this classic way where you would deal with threads and blocking but you actually not the status of the project it's still in kind of proof of concept so what I showed you does work I'm I just want to write to read me before I make it open source but it has test just no reason yet it'll be available very soon the channels are implemented that channels take care of buffering so you can actually rather than always blocking in a channel you can give it a capacity and the channel abstracts any kind of low level locking if you do need to do some locking because it could happen that you actually running in parallel on multiple threads as I mentioned put and take they actually return futures so you can use them in non go code so without that nice syntax trigger oh by the way so I should have mentioned this this operation here is the same so this means put this is the same as calling channel two dot put one except its syntax sugar that only works in the go routine because we're not returning a future anymore we're actually returning the result same goes for the read yes yes so as I mentioned put and take actually return futures but there's syntax you're on top of it saying goes would select it's a little bit more complex but it's also just a future and we're relying with piggybacking on Scala async to create this illusion that we that we're doing dealing with synchronous code and of course yeah there's a bunch of syntax sugar on top of it so we can just looks much nicer the roadmap is which is very early project but there's some features fundamental features that are implemented one of them is we need want to be able to close channels we need to deal with error handling I just showed you the happy path where the way we're using Scala icing is generating futures and relying on Scala async so we have a macro that generates a macro that then gets expanded so for proof of concept that's fine but ideally what we would want to do is hook into sociology creates the state machine of your program graph hook into that directly right because we can't do that we're a macro but that would need some a little bit more I need to explore a bit more of how we can hook into that potentially I've heard there's a project called Scala co-routines that might be able to work in conjunction with this then some other features like I want to be able to select puts so that I can wait until one channel is available to write on it also supporting buffer policies you don't always want to barking about but sometimes you can say like oh you know if I receive an extra result drop the first one drop the last one in my buffer you never want to have an unlimited buffer because that's memory but some other kind of policies community of course then some API improvements so I would love to have an alternative to symbols I modeled the symbols because I copied the closure async closure async used essentially these same symbols so I just like copied their their convention but I think couldn't a kiss is clearer another thing is this wildcard I'm not a fan of having these imports it's just magic I'd much rather import explicit things currently it's not possible because we need in an implicit that gives us some of this syntactic sugar but I'd like to see if there's not a way we can come up to monitors differently finally some refinements on types that we can pass channels as a read-only you're right only instance to other methods so just to summarize here what what does this project do is we've replaced threads and queues with conceptually lightweight threats and queues so programming model is again the same so it was all about reducing the complexity it's you can program with these familiar concepts but you get better concurrency much more fine grained things and the environment is going to take care of running it around Scala jeaious and the environment you want and it's all done in the library it's not in the language so go for example they designed the language so they could integrate this green thread lightweight threading channels abstraction very nice link to the language Scala fundamentally was developed to run on the JVM the JVM uses threads so we can't really do build that into the language itself so it's really nice that we have the power to do it in a library though and again so to summarize we've just added a layer of indirection so you solve problems finally I want to say there's of course other approaches to concurrency there's a might've heard of actors actors and communicating sequential processes are very similar they are opposite in many ways but that you can model the same things actors are named entities you send messages to an actor which processes they are anonymous you don't give a name to a go process so I just wrote go the message path on the other hand on an actor is anonymous you don't say how message is going to arrive to an actor you actually send it to this to an actor just goes there where as channels again they are in this case channels are names I'm writing to this channel sending messages is fundamentally non-blocking in actors so actors have a mailbox that's unlimited you can keep sending them things whereas channels they are have a bound bound buffer within and that means you can use channels to serve as a synchronization point you can actually make your code stop or again look like it stops while while you're awaiting a result this makes logic reasoning about things much easier of course on actors you can build that thing on top of it as well by creating these protocols saying okay I'm gonna send you a message if you have memory respond please and and you could you can emulate that on top of actors this is simple kind of what reactive streams do and any of these streaming library there's this whole process of asking like hey do you have can I get a new element this whole protocol to to get back pressure whereas in channels you have back pressure built-in finally this is quite an advanced and powerful concept and in the spirit of keeping things simple and making it easier for other people to understand because that's like the whole reason this this library is built is well first if you can get away with it just you know write a synchronous watch that's the easiest you can do of course many times especially in server applications you don't want that in that case futures and promises with Scala async perfect if you do need to then model recurrent events I would suggest well you could give this project a try or any of the other concurrency libraries and finally or if you really really really have no choice then you can consider callbacks but those are like don't try to avoid them thank you for listening I will make the project open-source very soon if you want to check it out I'd love to get more feedback on it [Applause] what what is the approximate ya think my I don't know if Mike's widgets so super quiet is like what would the performance comparison with go be it's a bit too early to say it's really just a proof of concept first of all the compile time performance is presumably going to be pretty bad as it stands because I just double macro expansion right but I can imagine the runtime itself is yeah and I don't know it's it would just run as fast as promises and futures essentially so I can't speak to much of it but one of the key things is I think to make this actually useful is we would need to I'm gonna try to try to come up with a way to actually build libraries that kind of you know use this may be some networking libraries that try to take care of it because this is just like as it is it's it's nice but yeah I want the least amount of dependencies possible I want something that just requires scholar that when you version scholar comes out I can run this directly on it or as soon as possible of course yeah you make a good point you know like I didn't mention scholars ducats there are like many frameworks that offer similar concepts all these channels right but this is this should be a more low-level thing that can still model the communication so this is amazing but like I'm trying to compare it with five couples in popular there was a problem to pass a tube through a channel so that the same agent like if one nation since a burner we're like a B so this net receiver will receive both will be not like the couple is yours this is much because those kids even walk so maybe exactly sure sending a tuple over channel yes there are five centers under five issues and if one sends two values so we want the same two values like another sensitive to the same two values to be received by exactly one consumer it's not built in like you yeah it's multiple readers multiple writers you write it you don't know which writers gonna get it it's like there's no yeah you have to model madness because it's very little any further questions all right well thank you [Applause]