Scale By The Bay 2021 : James Douglas, Live coding session with James Douglas

Scale By The Bay 2021 : James Douglas, Live coding session with James Douglas

Recording: Scale By The Bay 2021 : James Douglas, Live coding session with James Douglas

all right thanks indeed uh yeah so there's a poll in the discord channel uh basically for this session uh this is kind of an impromptu session so bear with me as i wing it but uh we're gonna be building a toy uh effect system in scala and uh there are a few options we can do monad transformers or fremon ads or tagless final or reader um looks like reader is the winner uh and uh that's my personal favorite so i'm happy with that so so let's do it um i'm keeping an eye on discord uh so feel free to drop any questions anytime uh this is very very informal um otherwise i'm just gonna be kind of talking through uh what i'm writing also in the interest of time since we only have 30 minutes i'll do rather than just making everybody watch me type in real time i'll copy and paste little blocks of code and then kind of explain uh each one as i do so um to to set this whole thing up uh we want to build an effect system and really if we back up a little bit we want to build a functional program that is itself a pure value um and our effect system is what will take that pure value and run it and actually make it do something so again in our little toy world we have to have a toy program and so our toy program is going to look something like this and by the way how is the font size i think you may need to zoom a little bit yeah that's brilliant let's see can i move this down it does not look like i can ah there we go perfect okay cool so here's an example of a program an imaginary program that we want to be able to write we can't write this today this wouldn't compile but basically we want to be able to produce programs that are pure values that represent a sequence of imperative instructions really um to to take so here's an example um called en program we don't know what the type is because this is imaginary but we want to be able to write in this style so here's a program that says first i want to read sorry first i want to write so that's probably printing to the screen a string so asking user for their name then i want to read probably from the keyboard a line of text and that'll be the person's name and then finally i want to write hello name back to the user so very very simple uh i o program but we want to write this without doing any actual like print lens or system.in.read or anything like that no side effects we want this just to be a pure value and then to to add a little bit more to this make it more interesting here's the exact same thing but in a different language so again we're writing reading and writing again but in a different language um and then i'll add one more program here that chooses between them based on an environment variable so the first thing that this one does again just wouldn't even compile but if this were if this could work what what this does is first read the lang environment variable um and if that starts with es uh then we'll run the es program um otherwise we will run the en program so each of these programs would probably have the same kind of a type it'd be a program for some kind you know type program that we need to create of unit so we're printing to the screen reading from the keyboard printing back to the screen and not returning anything interesting at the end it's just a unit so that is the setup that's the framing of this that's what we want we want to be able to write programs that look like this and to do it we're going to need an effect system so in other words we're going to need a way to interpret these interpret this code as a bunch of effects and in this case uh when i say effect i essentially mean computational effects or side effects or i don't know interesting things for the computer to do and uh specifically in this code we have there are three essentially classes of effects we have an effect that has to do with writing information somewhere probably to the screen we have an effect that's about getting information from somewhere probably the keyboard and then one more effect that is about reading information from somewhere else in this case the um the just the system environment that the program is running in uh so we have three different effects so we're going to write an effect system using free mone ads that give us these three distinct effects that we can then compose into any kind of program that we like that again is itself a pure value so i'm going to leave this here but i'm going to comment it out because it will not work yet and then we'll actually start to build this thing up and i understand that the bottom of the screen might be a little bit cut off so i'm going to just scroll a little bit okay and if if we can get through this thing quickly enough we'll do two versions we'll do a from scratch version of this and then the same thing but using zeo uh because we're we're um the effect system that we're gonna use the style that we're gonna use is the reader monad uh which uh if you squint and uh and simplify quite a lot uh is what zo is uh okay cool so we just to start off we'll uh create our definition of a reader monad so uh this is a reader uh it's a pretty simple class it's got a map and a flat map so it's monatic and it has two type arguments and one field called run that's a function so this says that reader for some type e and some type a abstracts over a function that takes an e and produces an a and the common way to think of this is e is some environment or it's something that is needed for this run function to run and a is the type that it produces you may often see this uh as um thought of as dependency injection where e is the dependency that run needs uh e is the dependency that needs to be injected to actually make the thing run and produce the a so anyway it's it's no more complex than that it's just a wrapper it's a case class wrapper around a function of type e to a and then we have a couple of functions so that we can combine reader with other stuff so if we've got a reader which abstracts over a function e to a and we want to operate directly on that a we just use map and you know this is a very very standard pattern so we've got a function that given an a produces a b map will let us turn our reader ea into our reader eb i'll skip over how that works just in the interest of time and then flatmap is same sort of thing if we have a function that operates on the a that this reader would produce and when run it produces another reader that takes some other type of environment and produces some other type of output flatmat lets us apply this function to this function to give us another reader one little tricky thing here is that the environment type of this reader that comes out of the flat map uh is a subtype of the environment that we started with so it it sort of abstracts over the original environment type and potentially adds some more information to it and we'll see why that's useful in a second here so let's create some effects using the style our first effect will be about reading from the environment reading environment variables and we'll create a trait called has m and it just has a method and that gives us a map of string to string so this is our our environment so if we have this map we can look up any environment variable that that we like let's see now let's make that into a reader with this function so this is a function that says if you give me the name of an environment variable i will give you a reader which is just an instance of this case class so you give me a name of a environment variable that you want i will give you a reader that given and given some kind of class that extends has end i will give you a string so basically this creates a continuation that given an environment that is a subtype of hazm we'll be able to look up the environment variable that we're looking for and very simply it just calls dot n on our environment because that that must be a method of the environment if it's a subtype of has end and passes name uh and so the type of this thing is reader of e string where e is a subclass of has m all right keeping an eye on please ask questions in the discord as they come up and i'll keep an eye on that and and answer them as i can um okay uh let's see we're 10 minutes in cool so we're doing well on time um i'm going to repeat this pattern uh for read lin and write so here maybe i'll visually separate these uh just this is a little bit easier to follow okay so here's the exact same thing but instead of instead of an effect that gives us a way to look up environment variables this is an effect that gives us a way to read strings from somewhere and we call it has read len so anything that is a has read lin has a method readlin that produces a string when we call it and then similarly we have a function here to create a reader out of it so if you ask if you call reedlin this will return a reader that takes some environment who is a subtype of hazard lin and produ produces a string when provided that environment and the way that works is given the environment which is an e which is a subtype of has redlin it produces a string which is just achieved by calling redland on that e cool and then one last one of these and that will be our right effect so same kind of thing um except our um has right effect gives us a way to given a string write it somewhere and then it returns nothing interesting so otherwise it's the same so we call right we give it the string that we want to to write and it returns a reader that needs an e which is a subtype of has right and given that e it can call write and provide the output and so so far we haven't actually implemented the way that we're going to read from the environment or read from the keyboard or write to the screen um none of that has been defined yet that that actually is really useful for um for testing because we can uh wait until we want to actually run our program and provide the uh provide different implementations of those things uh so for example in testing it's probably not convenient to be reading from the keyboard and writing to the screen we want something a little bit more deterministic than that and this pattern will give us that option and actually we're going to need it since we're in we're in this scasty environment we actually don't have a standard in to read from so we'll have to sort of fake it okay so that is really that's that's almost everything we need so our next step is to write our program um so the thing that we started with that's commented out up here um going to paste that again but with types this time but otherwise it will be exactly the same thing so here's our first program our first program has two effects it has writing and reading from standard in so our reader the type of this thing is a reader and its environment has the those two effects so it's a reader that has an environment with has read lit and has right meaning to run this thing we have to give it a way to read and a way to write um and the return type of this effect is unit uh and that comes from here because we're not actually um doing anything uh with any of the data um it's direct it's just io uh this program exact same thing just uh different strings here and then this program gets a little more interesting uh so our our environment type gets even bigger so program uh is a reader and unlike these two programs its environment has three different uh sort of effects in its type um so for program to be able to run we have to give it an environment or you know an e uh that is a subtype of has m and has relent and has read write or sorry has right um and that way uh to be you know to run this thing uh we'll have a way to look up environment variables and we'll have a way to uh read text and we'll have a way to write text and i think that's about all there is to say about that oh yeah so here at write read lin and read and those are those functions that we wrote here so here's reid lin that's that's in those four comprehensions below here's write uh and here's read em those are the when you see these functions here uh that's what they're referring back to um it's these ones that we wrote so they're returning readers and we can use this in a four comprehension because a reader has a map and a flat map okay so if we try to run this it will compile and it will run but nothing will happen because we haven't actually defined a way to run our program we've only defined a way to build up our program but that's what we set out to do originally we wanted to build uh we wanted to represent a program as a pure value so this is just a this is a reader which is a case class and um and and that's really all it is so our program can't actually run we have to come up with a way to run it but if you remember a reader does have a run field in it so a reader is a case class and it has one field called run so we can call that run and we can provide it the e that it needs the environment that it needs uh so let's do that so we're going to take our program and it's going to be this one so we have three programs we have n program s program and program we're going to run this third one so when we say program.run we're just dereferencing that run function that it's an e to a where e is this type and a is unit um so uh so to run it we're it's a it's just a function and we have to provide it with an environment of this type and so that's what we do here so we're just creating an anonymous class that has that type so this is a new has end with has redland with has right and to make that compile we just have to override those three methods you know one for for each of these um effect types so for to read from the environment we just look up um we need to get a map of string to string that will represent our environment we've already got that it's provided by the standard library so we'll just use sys.n for that for readlin we want a way to get text from the user normally we would do this this will read from standard input that's not going to work in this case the environment so um well i'll just make it always return james um and there this is an example of you know we can change our implementation based on what we're trying to do in this case we're trying to run on ski but if we were running from uh from a test we might have a list of um strings that we want standard input to look like and we might have a data structure that we want to uh store into for when we write that we could then later inspect so for for writing we can use print because this will give us standard out down here and and that should be it so i guess if there are no questions let's give this thing a run and it's not interactive because uh you know i'm not typing anything at the keyboard because of this line but but here you can see the output so it says what's your name and that happened here and then it it read from standard in which just uh you know got this string here and then it wrote oops it wrote hello name which is uh right here and uh we can force uh testing the other path um by overriding our so instead of using the the system environment we can make our own so we'll say our system environment variables are lang is something like that should work and so now we see the other uh program being run cool so that's that's really all there is to it um there's not too much boilerplate which is part of the reason i like this approach we have our our reader monad which you can find in pretty much any um functional programming library whether it's skelezed or cats and then we just have to write these uh these traits that represent our different kinds of environments uh that correspond to the different um you know functional effects that we want um and then these sort of convenience functions to create instances of reader that match the right environment and do the right thing and that's pretty much it um this is way uh more simplistic than zeo but it's it's uh philosophically it's the same idea um this is essentially what zio does zoo gives us a lot more it gives us i mean completely aside from the runtime that it gives us uh it it's more than just a reader sorry let me scroll up here it's more than just a reader with an environment and an output it also has a third parameter so r i think is what they call the environment and then e is a is like an error type and then a would be the the non-error result which is pretty handy so you can actually encode the potential errors uh that could result from your effects directly in the type of the effect itself so we've still got we've got nine minutes left um so i think uh if there are no questions um we can do a really quick version of the same thing but using uh well using a little older version of zio because that's what i have on hand but before we do that are there any questions on any of this code well we've got it here and feel free to drop those in discord yeah very quiet uh cool well uh then i will let's make a new one of these so we're going to use uh scala 2 with this now kind of outdated version of zio extra spt ah here we go so we'll use an older xeo 1.0 is up to two point something actually i don't know if their 2.0 is uh is shipped yet might be a release candidate anyway 1.0 is kind of old but that's okay we'll still get we'll get the idea so because we're using zeo we don't have to define reader or any of that stuff so we can define our has-m which looks exactly like it did before uh but to construct that rather than creating a instance of our reader case class we're going to create a zeo so here's the signature that i was talking about so zio is like reader um it's the uh it's like the single effect type um in zeo and an instance of zeo has three type parameters it has the environment type that it needs to be able to run um that's exactly like what we had it also has the error type that it could result in um and it has the uh uh return type that that it will produce if there's no error um and then this is just the incantation to construct a zeo with uh with this shape okay and then i guess i'll just sort of quickly paste these other two because they're not really dissimilar so here's our read lin very similar to before and then here's our uh has right so this is exactly what we had before the only difference really is the way we're constructing the zeo versus the reader from before but our our traits look the same which is kind of nice all right and then that's it so now we'll do our our three programs that we had before and let me get this away from the bottom of the screen a little bit all right so this looks pretty much exactly like it did before the only change is instead of reader this is a 0.0 and we've added this third type parameter for the the error type otherwise this is the exact same thing we had before and then running it looks pretty similar with zio we have these runtime environments um uh because theo has lots of uh support for fibers and uh async and threading and all and parallelism and all kinds of good stuff so we'll get a um a runtime from zio and um so we'll just get the the default runtime and then we'll say i i want to run uh i want to actually run a and this is a function that takes a zeo and runs it so i want to run [Music] my zeo and provide is a method on a zeo on an instance of zeo that lets us inject the environment so provide is just like our run from before and then we have this extra layer of having the zeo runtime actually run the thing but otherwise it's the exact same thing we construct a new has m with hazard lin with has right and then we define our three ways to do those things and so just like before this is not going to work and that should be all we need so let's give that a try hopefully this works with uh with the zeo library i have not actually tried this before hey it worked what do you know so anyway that's a a really quick overview of what reader is all about and how zio is a very sophisticated uh well this doesn't show the sophistication but zeo is a very sophisticated extension of that idea what else have we seen we've seen that our this lets us represent programs uh uh functionally because this this program is just a value has no actual behavior and the behavior comes from here i think that's about all i've got are there any questions nothing on this card cool well um i think we can call it and i'll jump over to the spatial chat in case anyone wants to talk about this um i can also point you to some other examples if you'd like to compare this to tagless final or free monads or mona transformers or anything else uh just so you know people we have three four minutes so you can actually ask questions on this card before and someone is typing so hopefully we'll see something come up oh you scared them away oh that was lavanya okay it's about our previous one actually same friend is typing [Music] the question is that i said reader is my favorite and why uh that's a great question um the short answer is that uh i'll just go through the four um that well actually i'll go through six so monad transformers is uh just it turns out to be extremely noisy the code is is uh not fun to maintain uh it's just a ton of boilerplate and the type signatures are horrible and it's just really unpleasant to work with um fremonted is actually pretty nice uh but it's a little bit inefficient because it has lots and lots of allocation and garbage collection and there's lots of instances of case classes wrapping instances of case classes tagless final is uh a lot better than free monads even though they're i learned from sergey wenitzky who's at this conference that they are equivalent one is the church encoding of the other um tagless final so it solves a lot of those problems with the performance of allocating lots of case classes uh but it relies on a a higher kind of type uh pattern that's a little bit confusing especially to newcomers that i've trained in the past um but it it's benef to me one of its main benefits is um it's very popular especially in the type level ecosystem um so it's probably worth learning if you're going to use any of that stuff but if you have the option of of not doing it then it's it's a level of complexity that you don't need to worry about with reader and so i guess that brings me to reader i like reader because it's very very simple it's straightforward for me it's easy to understand and and it i would not use the reader that we wrote here in production um it's not going to be efficient and it's probably going to blow the stack um but the pattern is very easy for me to understand and i find it very approachable and it happens to be the way that zio works and so zio is on the unlike my my reader implementation xeo is very much production ready uh and has a lot of features that i like so i think in in part my answer to to why reader is certainly i like working with zio um but also i find that the simplicity and the the entry cost to be very low the simplicity to be high the entry cost would be low and then two other patterns that i did not talk about today are m and f and that's just because they're over my head and i i can't i i don't understand them yet so uh maybe next year you