SF Scala: Essential Effects By Adam Rosien

SF Scala: Essential Effects By Adam Rosien

Recording: SF Scala: Essential Effects By Adam Rosien

Good evening everyone. Welcome to SF Scala Meetup and Vancouver Scala Meetup tonight. Thank you all for coming. Tonight's speaker is Adam Rosin of Inner Product. He's going to be talking about Essential Effects. Over to you, Adam. Hi, thanks for having me. Um it's been a long time since I've given a talk because of, you know, the world

But um I'm very happy to do so. Um so my name's Adam Rosin and I work for a small company called Inner Product. Um we still I started out well, not not really, but back in the day I I worked with Underscore, who you might know. Um And that's where I met Noel Welsh and I kind of was the US version uh the US associate over here for Underscore, which is mainly some EU folks. And then we decided to make our own little consulting company here. So we do a lot of things like we do a lot of training, we do mentoring. So if you want um uh nerd as a service, you just sign up with us and we can we can help you fix bugs, develop your product, work on your in your staff. Um work along with your staff on projects

Uh and it's kind of what I've been spending a fair amount of time with um lately has been a new course and a book uh a called Essential Effects. So it's about how do we uh well, I'm going to I'm going to share basically the the what it's all about. You might have heard about effects uh through things like side effects. I'm going to kind of assume that folks are somewhat um exposed to effects and like Cats Effect, but I'm not really going to assume that people know too much. So it's not going to be a very advanced talk. Um I'm happy to sort of stay on and there's other folks online who could answer some of the the fancier questions. I'm happy to talk about that after the main main talk. Um but this is meant to introduce effects, why why we should care about them, uh what we need to know, and how to use Cats Effect to um take advantage of of some of these patterns

Um so, the big news right now that I am was furiously working on minutes before I actually arrived here, uh is that the book I've published I just published the sort of early access edition. So, you can go to essentialeffects.dev and you can buy it for the early access price. Um it has, you know, everything that I've been talking about. Uh there's a bunch of exercises. It's 170-some pages. I was really nervous to sort of put it out cuz you're like, "Oh, there's going to be some stupid mistake that I'm going to get embarrassed about." But you got to just sort of push the button and this this uh having this talk was a good um deadline to sort of uh force myself to to do it. So, so please check that out. Let me know what you think

Um So, here's kind of what I'm hoping to talk about today. I I'm I should probably get to all of it. Um it's not going to cover everything in in Cats Effect, but I tried to sort of figure out a good subset to be able to cover in uh maybe about 45 minutes worth of time and then we'll have some questions. Uh there's lots of other aspects, but these are the these are the big parts. Um and so, as uh Solaris mentioned, please please let me know uh if you have any questions or if you need me to repeat something or I say something wrong or something like that. He he'll he'll uh prompt me. Uh I don't know if I can see you directly. Um so, that's what we're going to do

We're going to start there. It is difficult to give a talk when that can't quite see your faces, so um if I say a bad joke, I'm going to laugh, so you don't have to. All right, there we go. See, I can hear it. I can hear the laughing. All right, so we're going to talk first about effects, and this is going to be a bit more about like what's the point of all these things? Why do we Why do we care? Um and since we're since we're functional programmers or we pretend to be or you know, we get paid to be functional programmers, we always have to talk about the substitution model of evaluation. So, this is this is a model of evaluation. This is not what really happens in the computer, but it's it's a way for us humans to understand it

Uh and sort of the typical way we teach it and talk about it is we have some sort of expression. You know, we usually use arithmetic. And you know, we have some expression like 3 * 4 + 2 * 5. This is an expression. And we assign the it to a value. And the basic idea is that we have these expressions, and oh wait, expressions are made up of others. So, um please internet gods, treat me well. Um we have this model of evaluation where we have these expressions, and they you know, they're made up of other expressions, and then we evaluate them

So, uh we say we simplify the uh the first one, the first sub-expression 2 * 4, that's 12. Then we say, oh, well, there's another sub-expression. 2 * 5, oh well, you know, that's 10. And we have yet another expression. So, there's a very mechanical process. Um and we get an answer that we can't is no longer an expression, but it's in a value. It's a value that we can no longer evaluate. So, this is a very simple model

It's not exactly what really happens in the computer, but it's it's used it's for us as humans to sort of be able to explain to ourselves something that the computer should match. The computer should get the same answer. We can teach people how to do this. It's a very you know, we we teach them the rules, how to you know, start on the innermost expressions or there's some rules around it. And it's uh it's so easy even a computer can do it. So this is nice, but unfortunately, a lot of what we do um doesn't follow this model. So when does substitution not work? When does it break down? So have a little think. Have a slurp of tea

When does uh substitution not work? I'll give you some examples. You probably have seen this before. Um So one example would be uh the sort of ubiquitous print line method. And that'll print something to the console. Uh if we evaluate that expression, it has type unit. So it returns a unit. And so when we're when we're performing the substitution we should be able to substitute the expression for what it evaluates. When we say that things one thing is substitutable for another, it means that they mean the same thing

But does printing to the console mean the same thing as unit? No, because if we just had a unit, there's no printing going on. So So this is where sort of the term side effects comes into into place. There's some sort of side effect that we're declaring uh is the is is embedded in the meaning of print line, but we can't really see it because when we evaluate it, all we get is this unit out and unit does not mean the same thing as printing to the console. Other examples would be um instead of writing sort of the outside world, maybe we're reading something from the environment, from the outside world. I might be reading a a line from the console, or I might be even you can think about it as, you know, reading some some bytes from a file. You don't know what's in the file. And so if you evaluate something that is interacting with the environment, it will evaluate to whatever's in the file, whatever you typed on the console. Um but you could have typed something else

You could be reading a different block. Um so the meaning of this uh expression here, like read line, is context sensitive. It depends on the state of the outside world. So you can't use substitution to understand what this uh what this function is going to do, because you're always going to get something different, or you will likely get something different. And similarly, um you know, that carries on to the idea of some, you know, mutable value. So if you're if we're interacting with the environment, if if sort of this is var is out there, acts accessible to our uh expressions that we're programming with, that sort of that mutable variable can change. So if I use substitution to evaluate the expression, what it evaluates to really depends on the current state. But as we all know, the current state could change, the environment can change

And so substitution just does not work. And you know, you can think of four more examples. So so why is this important? Uh we well, you know, if we if we have this world of, you know, arithmetic and perhaps other things where the meaning of some expression uh is maintained when we evaluate it, if they mean the same thing as is they do evaluate to. And then we have this other sort of set of things that break that property where if we perform substitution, they don't they don't mean the same things. Um how do we deal with that? Well, here's another example that's perhaps a bit more realistic. So, this is the this is a the future type from the main Scala library. If we had a function, uh if we had a a future, you know, that just printed to the console again, so we can sort of say, "Aha, I know something sneaky's going on." Um well, if we if we call it if we call it twice, we could implement that that twice as, you know, do print and then flat map says, "Okay, well, after I'm done, perform some perform the action of an of the future of another future." If I use the same future, when I run this, I get the I get the output hello world. If we unpack this a bit, if we perform this substitution process, um and I'm going to sort of walk you through it, we can see sort of what might be strange about about how future acts

So, uh substitution is all about substituting expressions for their definitions. So, if I substitute print for its definition here, I get that. And then if I do it again, I substitute the the print for its definition here. If substitution holds, then this new code piece of code um should mean the same thing as the original piece of code. And if I perform if I then uh run that thing, so here's my substituted expression, if I run that in your in your own mental simulator, you're like, "Hmm, what does this what what what effect does this have?" Uh well, it prints hello world twice. So, there's there's uh there's obviously some difference between what future uh the first expression and the second expression that breaks substitution. Um so here's a nice little picture from our our our internet friend impure pics. Uh there's something going on with future

Well, so the problem is that these side effects, they happen too soon. Uh so how do we fix things that happen too soon? We don't make them happen too soon. This is easy. Aha. You know, they go to the doctor says, "Oh, you know, my arm hurts." They say, "Well, don't move your arm." That is the way of punctual programmers. We do the exact same thing. Don't want side effects to happen? Don't do side effects now. So if we instead instead of having future which has this problem of of running things too early we can sort of create our own uh I'm going to call it my IO just distinguish it from sort of the real IO which is a little bit fancier

Um the idea here is that we're going to capture whatever the effect is, whatever the side effect is, I should say. We're going to capture it in a function. So we're not going to perform the print line or whatever we're going to do now. We're just going to capture it in a a zero arg function. And it will eventually produce an A, but just not now. So So this is the thing that's actually going to uh encapsulate our side effect. So we're going to kind of wrap it in this nice little bubble. It's called unsafe because it's meant to sort of give you a warning as a programmer

If somebody calls unsafe that implies that substitution does not hold. And the idea is that we're going to you know, isolate this unsafe thing. And everything that we're going to do, any other kind of method that doesn't have this unsafe keyword, we're going to say, "Oh, well, that that's safe." So, it will respect substitution. So, once we can construct one of these IOs and delay the actual side effect, well, we could just call unsafe run again. That would run it. But, but we don't want to do that all the time. If we wanted to uh perhaps transform the output of our IO into a different output, that would be a map call. So, we can produce from an from a my IO my IOA, we can produce a my IOB if we give it an A to B function

And so, we could just construct it. Uh we construct a new IO IO new my IO and when when it's run, so this is the unsafe run the body of the unsafe run for the new IO, when it we first run the original unsafe run and then we transform it with our A to B function. So, we're still not running anything. We're just constructing a new my IO. Nothing happens when we call map. And similarly for flat map, we can construct a new my IO that when it's run, not now, but later, uh we first run the original, we get an A, we call F on A, that gives us a my IO IO of B. We're supposed to return a B, so we call unsafe run on that thing. So, this is where you can see the sequencing happen

First, I run that first effect, that produces a new effect, and then I run the next effect. So, this is a little safe container. Inside this container is is the unsafe code that uh doesn't respect substitution. Uh but once we wrap it in it, we can do maps and flat maps, you know, uh and everything becomes safe. So, sort of our our checklist, we want to make sure that we're not doing side effects now. We want to do our side effects later. So, are we calling unsafe run in the constructor? No, we're actually we're capturing some effect here. Are we calling unsafe run during composition? So, during maps and flat maps and other methods? No, we're always constructing a new my IO

So, we're This means that substitution can be maintained. Uh so, if we take that sort of twice program instead of using future, we use my IO. Uh we would construct our my IO, we perform our flat map and do it twice. Uh Now, at this point, nothing happens. This is good. This is like, okay, I'm just describing what I want to do. I'm not actually doing anything yet. And then you have to explicitly say, "Go." You have to explicitly say, "I'm going to do my unsafe thing that doesn't support uh substitution anymore, but that's okay

I've constructed my program and I don't need to do anything else now. Now, I'm ready to go. If I perform sort of the manual substitution, I replace the uh the expression with its definition. If I do that here and I expand this into its definition, uh I get this flat map of the two my IOs. If I expand the flat map, you know, here's the definition. You do, you know, you just sort of mechanically just I flat map turns into I'm going to do this thing first. I'm going to call I'm going to do my print line first. I call F, it produces a new IO new IO

I call unsafe run. And eventually, everything just unwinds and print do my first print line. I then call unsafe run. On the second one, I get an IO out. I print that one, and I get two hello worlds, which is what we'd expect. We do have a question, Adam, if if it's a good time for me to interrupt you. Sure, definitely. Uh what are the performance implications of effects? Are the thunks stored in the heap? Is there a major trade-off because of this? Hm

Um I see Daniel's uh uh video went out. Um I would So, there's lots of ways to answer that. Um I think I think a safe general answer would be that um it usually doesn't matter based on what you're doing. So, in There's sort of a nor One of the usual arguments is that oh well, a lot of what we do in our programs uh is sort of IO bound. We're talking over the network or parsing uh JSON or I don't know what, but you know, we're we're not really the cost of doing these flat maps is not too much. Um you know, that that obviously has a giant asterisk next to it that of course you know, it it's um it's not the same as just inline functions and sort of normal optimizations, but um I you know, I can I can safely say that it gets pretty close. Um there's certainly different pathologies that you have to know about. Uh and I will probably um uh refer people to chat with somebody like Daniel or or or or Mike uh after the after the talk

In general, but in general, it's not that big of a deal. And the JVM's really good, so there you go. Thanks for the question. So, so why am I going through this very tedious exercise? Um so, like we have the substitution and look, oh look, we're substituting and everything works out. Um uh the reason it the reason that this is really important um so, here's the match. Phew. Um here's the big summary that I I I'm not quite going to read, but um the important part really is that uh I would say it's sort of part of this first point. Um the reason substitution is important and I didn't I didn't mention this sort of uh you know, the the the varying the word that other people are you know, not other people, but the words uh that is often used as referential transparency

What I'm talking about is referential transparency when substitution holds. So, referential transparency substitution is a great property to have. Because it means you can always perform you can replace some expression with its definition. And it it and if you do that nothing will change. That the behavior that you see will remain unchanged. So, it means that you can always do an inline uh you can you can expand a definition with its implementation. You can literally take the code and replace it yourself and nothing should change in your program. And also the opposite

You could always you're always able to refactor. So, if you have some some uh sort of sub step in a in a in a program or in a method, you can always factor that out into its own method and nothing will have changed. That's a really great property. If you think about some of the hairy code that you might have encountered or even wrote one you might think of examples where you could not do that. If If pulled out some piece and factored it out into another method, it won't work because you're referring to some um uh state or something outside of the lexical scope. And then you have to you have to pass it in. And sort of all you're you're relying on all these things. But if you have this notion of referential transparency of of allowing substitution, uh you really don't have to worry about those things

And it really helps it sort of reduces your cognitive burden a lot as a programmer. So So that's why this is uh really important. So that's kind of the basic of effects. We want to make sure that we're not doing these side effects because if we do side effects now, we can't maintain substitution. And And substitution gives us all these other great things. So instead of doing these uh side effects now, we do them later. And so we end up with something where we can compose our program. So we can do maps, flat maps, applicative map ends

Uh and we can compose these things. And no matter And all of those operations don't run anything yet. We're just building up a larger and larger uh program. Then finally, when we're done composing, then we call this unsafe run. Uh and maybe even we do it, you know, as late as late as late as possible, which means like in main, you know, when the program starts. That's the That's sort of the best place. So then you know, oh I I've I've composed my program together and now go. So that's the idea of effects

Now I'm going to jump into uh the real IO. So instead of this sort of toy, um the real IO in Cats Effect, a super library, a great library. It's so useful. Um it builds on uh the whole type level ecosystem. It builds on Scala. It provides so much. I can't show all of it, but I'm going to try and give you just sort of the you know the quick cheat sheet of how you might use it. Um So it's it's a data type

Um you can think of it as one way to think about about it if you haven't used it before is everywhere you use um future or perhaps try, you could just replace it with IO and your program will be better. Um because it's safer. So we have to get into IO. Um so the most common method that you might use uh would be the IO delay. It's alias is just IO apply. So you can just say IO parentheses and give it an expression. So it this is this is where the delay happens. You can see that this is a by name parameter

That's what this little arrow means. Means whatever I pass into the IO uh factory method there, it's not going to evaluate it. With normal function calls, uh an expression passed in as an argument to a function is evaluated now before the function's called. With a call by name argument, uh this little arrow, it says don't evaluate it now, evaluate it later. And that gives us nice semantics that we want. So So this this will capture the side effect and produce an IO that when it's run later, when you unsafe run something, uh eventually you'll get an A out of it for whatever the expression produces. Um you might already have a value. Uh but you need to be you know you need it to work in sort of the IO ecosystem

So one way you can lift it would just be through the pure method. And you would give it an already existing value. And that again would give you an IO value. This is more dangerous because you don't want to do a side effect when you call uh when you pass an argument to pure. That sort of defeats the whole purpose. If If you ever perform a side effect, you want to use the IO apply or or or delay. Pure is just about values that have already been evaluated. These are true values

So, that's how we get into IO. Uh there's usually sort of three steps. You get in, you do a bunch of stuff within that world, and then you might get out. So, So, once you're in the world, these are sometimes called combinators. Uh they're they're about transforming the IO into into another IO. So, these are are very familiar things. Uh these would be sort of the most common combinators. You can map, you know, this would be IOA give you an A to B to an IOB

Uh map N is the applicative map. So, that would take multiple IOs uh and allow you to combine their results into a single IO. Flat map or, you know, typically you would use it in a for comprehension. Uh that allows you to sequence uh IO effects. We'll We'll call these things effects now. So, an IO An IO value is an effect that produces a value of type A. So, flat map lets you uh execute one IO to produce yet another IO. Uh So, they're connected together

And then there's a whole slew of air handling. So, um if there is an error, there's a notion of error with IO. So, you don't see it in the signature. So, there's only one type parameter for IO, and it's about it's about the value the type of the value that produces if that computation is successful. So, if I'm successful, I will return an A. Um but you could you can imagine some uh some expression that you've captured here that say throws an throws an exception. Uh there is a notion of error in IO. So, uh for you know, map, map then, flat map, these would stop

They would not return a new IO if they uh if there was a failure. You know, certain as would uh a future, as would a try, any sort of other kind of uh monad. Um but, we could do things like recovering error from errors. So, we can handle an error. We That would transform the error case. The error case for an IO is a throwable. So, we could choose how to handle that throwable. We could produce a new IO

We can produce a value. We can transform that throwable into something else. There's a whole bunch of variations here. But, we're still working within IO. So, these are sort of IO to IO kinds of methods. So, we go into IO. We sort of play around in IO. And then, when we're done, you know, usually at the sort of start of our program to start the whole process

Um most commonly we'll use these They're all unsafe. These are sometimes called eliminators. We take the IO and we rip it apart. You know, we take an IO and produce the A inside of it. Uh this is run sync, so it's unsafe because whatever we're doing is performing side effects perhaps. And uh substitution is not involved, so that's a little warning to us. We're running it, and we're running it synchronously. So, if you have an IO of A, it's going to produce an A

You can imagine, and you'd be right, that there's a run async, and you know, that would run it asynchronously. There's other options. And sort of as a beginner, or someone new to library, uh and probably, you know, it's likely that you're using future in your in your code. Um you can convert an IO into a future. So, there's like an unsafe future, unsafe to future method. So, this is sort of tears that IO apart. So, these are the basic steps. Um, there's there's there's not much to to show about it, but I'll I'll be giving more and more examples

This is this is how I build it. Now, since it's a really good idea to only to perhaps maybe never call Well, someone's going to have to call that unsafe run at at at some point. So, the smart folks of Cats Effect says, well, let's let's make our own application class. So, in in Scala, there's the app class that you you might typically use where there would be the main method and and things like that. Um, but so, there's an there's an IO app. And sort of it's version of main is this run method that you have to you have to um um, implement. And so, the entire program, when you run the program, that is supposed to return an IO. It's an effect

It's some effect that's going to run. And it eventually when it it eventually is going to return some exit code that says, you know, I I succeeded or I failed. Um, and so, you can imagine implementing this run method, you return an IO, and you'll remember that IO is nothing has ever run yet. So, that must mean that IO app itself is going to run that IO. So, it it it sort of takes the pressure off. You no longer have to call unsafe run from anywhere. If you implement IO app, you just implement run, return some IO, and it's IO app's job to do, you know, to do the dirty business there. So, this is this is this would be like an example of a, you know, hello world

Um, I have some hello world effect. It does it prints hello, it prints world, and the definition of uh Oh, that's not right. That should be a print line in there. Um Aha, bugs. Good thing. If we use substitution, we can figure this out. Uh this would uh you know, print hello world to the screen. So so we're just composing programs, composing programs

Nothing actually happens yet until we return this IO, and IO app says go. All is well. Um, so I went pretty fast. But um there's not many unexpected things. Once you sort of learn to construct IOs, uh you know, passing in uh things, and then using sort of normal uh um combinators, you normal constructions, you know, using for comprehensions, you just build up larger programs, and then uh the IO app itself uh will then run them for you. So, that's like the super fast uh you know, cheat sheet for IO right there. Okay, so that's fine. You know, I've given everybody more work to do to convert all their code from uh future to IO

You went through, you wrote maybe you wrote a program, maybe you're old school like me, and you just did like a said script, and you just changed future uh to IO everywhere, and everything just compiled. Um we get more from IO. So, IO IO lets us do more. So, the next thing we're going to talk about uh is sort of the the real workhorse of it. Uh it's about parallel execution. So, um in when I teach this course, and uh you know, the this the course that these slides are based on and uh and now you know, when I when people read the book, part of the difficulty with parallelism you know, we have these multi-core computers, we're running code in parallel. How do you know that something's running in parallel? You kind of have to actually do a side effect to say like you know, we're running on the JVM so we're using threads. So so it you really want to know, am I using am I using these threads or do I have a single threaded program? So in the examples that I show, I'm going to take advantage of a little extension method that I cooked up

It's just going to be this little debug method and I have to say I totally stole it from the FS2 folks. Cuz it's really nice idea. It's like, I don't know what to do. Just add a debug in there and and you're done. So it doesn't have any actual effect on it sort of decorates an existing IO and it doesn't change what this IO does, but it just adds a little bit of printing. So it's going to print the current current thread name along with the value that was in the IO and then just yield the value so that normal processing continue. So the idea is that if I'm just trying to figure out the lay of the land, if I'm trying to understand the behavior of parallelism with respect to IO or something like that I want to be able to see where my code is running. So I have this little debug helper

Um So here's a little here's a little app and uh all it's going to do is uh it's going to do a map end of two effects. So hello is an effect that's going to debug the value hello. So that will print hello to the console, and and world will will take this value world and and print it to the console as a side effect. The values inside will carry on. So, when we compose with mapN two effects, we get their output values if they both succeed. So, hello would produce this H, and world would produce this W, and I'm creating a new effect out of two that's going to produce this new value, and I'm collecting them together, and I'm going to debug that. So, if I uh if I run that program, you know, I I can see my um Oops, these are the ones I'm supposed to be there. I see my hello, I see my world, and I see my hello world

So, no no mysteries there. I'm printing to the console. Uh but, the thing to notice is that when I do my when these IOs execute and when their composition executes, they're all running on the same thread. They're all running on thread zero. So, uh what I'm intending to demonstrate here is that um IO as it's uh on its own doesn't have a notion of of running things in parallel. It just runs things, you know, sequentially. So, there's some other way to to access parallelism. Um and it's pretty easy in some senses

We just add par in front of our mapN. Just add par. There you go. Oh, my hand's not appearing. Um so, with a par mapN, it's a different method, and it it accesses different behavior. So, if I have the same program, change mapN to par mapN, there are a few requirements that need to be there, but if in in this example, nothing else needs to change except those three three letters and a capitalization. Uh if I run that, I can see with my little debug helpers cuz they print out the thread. Uh, ignore that little one

That's my my mistake. We can see that hello and world are running on different threads. This is so amazing. Uh, and then finally, the the the combination of them, you know, is running on some other thread. Well, it it runs on whatever the last one is, but it it's really a detail. Um, so I Now, now with a little bit of help in order to see kind of what's going on, uh, we can see, oh, these things are running in different threads, and I can sort of just, you know, that means that they're running, um, in parallel. Aha. So, what's what's really nice and I I don't know

It It's kind of nice when you just change three, you know, three and a half characters, and you know, you just sprinkle on parallelism. It just It just kind of works. It's It's really great. Um, so that's kind of like the simple recipe. Oh, these And it's it's also sort of implied in the structure of the computation. When I When I have these two computations, uh, I also noticed that world happened before hello. Um, and that's because they're running in different threads. They could get scheduled in any order

Um, so these these, uh, these all just, uh, just happen. Okay. So, we went from map end, which is just sort of a parallel uh, a sequential notion, uh, has at least sequential semantics, um, by default, and we changed it to par map end, and now we have parallel semantics. Um, if you recall, um, error handling, error handling in a sequential world, even with an applicative map end, um, goes, uh, you know, there's some order which it it executes in and it happens to execute left to right. Which is a sensible default. But if you were if you're executing this code, here's a little bit more complicated thing. How does error handling work when there's parallelism? So here's three effects. There's the okay effect which you know, has no errors

There's the KO, the opposite of okay. It raises an error. I probably should have mentioned that one, but so just like you can create an IO a successful IO value from from just a regular value with pure you might want to take a a a failure and lift it into an IO. And so the failure type for an IO is an is an exception. So you're basically lifting an exception into an IO. You can use that use the raise error factory method. We say, oh, it would have produced a string, but it actually threw an exception. So KO1 is going to throw a an O and KO2 is going to throw a nose

And we're going to have three little test cases. So in all of them you know, there's an error. There's a KO. So we need to think about what is the value what do these these composed effects evaluate to? We know we can assume that since there's an error, you know, we're running these in parallel, we're running these in parallel, we're running these in parallel. We know that if if any of them have an error, then it's fair to assume that the composed effect will error. And one one way you could deduce that would be for parMapN to succeed, it needs to applies this function. This function can only receive arguments if both of the its input effects produce values. But we know that KL1 does not produce a value, so essentially the value of this slot would never be produced

Which means this function would never be called, which means we can't uh we have to produce an error. We can't actually successfully call this function. So we know that all all three of these effects here will fail. And the one question would be, well what's uh you know and with our little debug helper here, you can see, okay, well, what uh what output do we see? So going through them, uh here's what you see you typically see, I should say. It's uh it's it's a bit of an exercise to think about why it might not show exactly what I I put up here. But um so we see the output of okay, that was like hi. Um and then the failure happened. So the failure doesn't produce any output

And then the value it we're getting a debug on the whole effect. Uh and it prints it says uh and if I just back up 1 second, um you'll see that in my program here here's my IO app, here's my program. Uh I'm using this combinator here. It's one of the error handling combinators. It's called attempt. And what attempt does is uh normally, if there was an exception thrown or a an error raised for an IO, uh all all execution would stop, you know, there's nothing no value to produce, so you can't keep you can't keep going. But what's what's interesting about attempt is it it sort of takes uh it takes an IO of A and will give you an IO of an either. So, if there's an error, you'll get a successful left of throwable

And if the if it succeeded, you'll get a successful IO of right of the value. So, it's a way of sort of ensuring this effect succeeds uh and the value represents the fact whether it whether the computation whether an exception was thrown or not. Uh so, that I'm using that in order to run all of these effects which I know to fail. So, we see in the end that E1 is a left and um my screen here we get it we get an O. Okay, that's fine. Uh if we reverse these arguments, uh you know, you'll get something similar. Maybe you'll just see the O. We know that this is the value of E2

It failed. Okay, fine. Okay didn't have time to run in this example. Maybe we got lucky. And then there's the example, well, what if you have two errors? Which error do you get? Well, it's non-deterministic. We don't know. Both of these uh both of these effects will will raise an error. And really it's just whichever one happens first is the one that the composite effect will have will uh will become

I don't know the right verb, but will assume that value. So, it could be that we get an O because KO1 failed first or maybe KO2 failed first. It's a bit non-deterministic. So, that's the that's one caveat to to to to um to know about. Just as mapN requires all of the inputs uh all the inputs to succeed in order to produce an output, um the same um conditions hold for running things in parallel. And because we have to stop, it's it be it's a it ends up being whichever air happened, you know, first according to the system. Are people still there? Tell me, Sol. There are

You got 63 people on switch right now. Okay. Good to know. Just Just needed to check. So, we can par map in, uh you know, just sprinkle parallelism. There's a couple other forms that essentially do uh that launch, you know, you could think of them as running things in parallel. Um there's the ubiquitous traverse, everyone's favorite method. It's either map, flat map, or fold

Oh, and you know, most folds are probably just traverses. So, there you go. It's The answer's always traverse. Um so, instead of traverse, we say par traverse. And that will um So, the the form of a traverse is you typically um would have a collection, you know, this is like the work that you want to do, the set of work items. So, I made this tasks, and it's just a bunch of ints. And then, what else is involved in a traverse? There's a a set of things of work, and then there's a function. Here's my function uh that does one unit of work

So, given an int, I'm going to do I'm going to do my uh uh I I I'm going to process it, and my result type is uh happens to also be int. It could be whatever you wanted. Uh and it's just some dummy kind of thing. And so, given a bunch of work and a function that produces an effect for one piece of work, the output of traverse would be one effect that has the lists uh of all the outputs. Uh so, I traverse over the collection you is a typical um typical example. You give it a function which does that work. And then I'm going to, um, print the output. And so I get, uh, a bunch of, uh, output

And you'll notice that it happens when I traverse, I get a bunch of different threads. I get threads 1 and 2, 3, 4, 5. Uh, I have eight threads available on my JVM. So this lets you do a a whole bunch of work. And then the normal, uh, cousin, sibling, who knows, maybe sibling, uh, of traverse is sequence. Uh, and that's where if you have a list of effects, you turn it into one effect with a list of results. Uh, so that's what pa- that's what sequence and it's, uh, it's it's, uh, sibling, it's evil twin, I should say, par, par sequence. So anywhere where you have, um, map, traverse, sequence, you now have access to these par versions

And under the hood they get scheduled, uh, and they'll run on top of the existing threads as as much as they can. Um, that is like the super basics of, um, parallel execution with IO. Very similar to, um, future, except, uh, with future things are a lot more implicit. You implicitly have parallelism. IO, on the other hand, you have much more, um, explicit parallelism, uh, with this different set of, uh, combinators, different set of, uh, operations that you can do with the par. Okay. Any questions? All right, I'll keep going. How's my time? All right

Yeah. Not on Twitch. We're all good. Okay. Okay. So, there are kind of two So, that that we we we talked about effects and substitution. We love substitution. We have our IO which which lets us perform substitution and understand our programs, refactor them

We sort of get this parallelism. We just just add parallelism. Life is great. But then it gets a bit more complicated. We start to We start to run everything in parallel. Why not? We have these great computers. Just add more parallelism. Everything's going to get faster and faster

It's going to get you know, ludicrous speed. Well, how much parallelism can we really get? So, if I write a little program, you know, I I figure out how many CPUs I have in my JVM, and then I do a par traverse, and I give it I give it more work the amount of I give it more work than CPUs. So, I'm going to produce, you know, twice as much work as I have CPUs. Um and you know, let's let's see what happens and sort of debug our results. You know, if I have Do I What what happens? Let's see. Do I even print it? Oh, I was going to switch screens, but I don't think I can switch screens right now. That's okay. Let me tell you what happens

We kind of saw it in in the previous par example. Um My my computer and my JVM, they only have so many threads available. And it turns out that I have On my computer, I have eight CPUs and and I get and when I have these little debug statements that sort of say, "Oh, I'm doing work. What thread am I on?" Uh I will see output that says, "I'm on thread zero. I'm on thread one." all the way up to thread seven. So, I I I really don't have access to more than eight threads. Now, it's not true that a computer uh can, you know, so one question we can ask is, can can a computer do more than if I have eight CPUs, can my computer do more than eight things at a time? Um and it's kind of a subtle question um because we could we could say, "Well, obviously, you know, my computer's doing more than you know, my computer right now is definitely doing more than eight things at a time." Um uh you know, I see UI updates and and all sorts of network things going on that I don't see and my I I don't know what, you know, but there's lots of things going on. Surely, there must be more than eight things happening at the same time

Um but it's kind of a a a matter of of what you mean because I only have eight CPUs, so really only eight CPUs could be doing something. So, if what does it mean to have be have something happening? Um and so so um so kind of what I'm I'm I'm trying to lead you towards is is the idea that there's some sort of abstraction going on here. Um and the idea of a thread, for example, you know, we could just program directly against uh some super low interface that says, "You run this run this IO, for example, on a CPU." And you have to give it the CPU that you want it to run on. Um and you know, I want to run run it on CPU four. And then you got if you try to if something else tries to run on CPU four, well, it would have to wait till it's done. So, you know, the the elders in our field said, "Well, this is silly. I don't want to have to say what CPU I want to use. I'm going to just make this I'm going to abstract over the number of CPUs, and so I'm going to have sort of this logical notion of of computation, and we'll call it a we'll call it a thread." You know, I'm I'm making up this history, but this is this is one way to kind of summarize it

Um and you know, you can have as many threads as as we allow. Uh and so you as a programmer can think in terms of threads. You and those threads would will all sort of compete for the use of those underlying CPUs. So, you don't have to think about CPUs anymore. You have to think about threads. Okay. That's good. Um but how do those threads Who says how those threads, you know, use that underlying CPU? You know, it's the JVM's job to do that

Oh, okay. And there's some rules in there and so on and so on. Um but essentially we have this this layer of these layers of abstractions that let us have access to certain concepts, and certain concepts are hidden. So, the idea is that whenever we um represent some computation, if we have an IO, for example, and we're doing a say par traverse, um this is also an abstraction. Um and and a simple way to think about it would be that um you know, all of these tasks, you know, when I run this traverse, I get back one IO, and when I run that one IO, there's going to be in my case 16 tasks that are all trying to compete for that those eight um you know, eight CPUs, say. Um whose job is it to make that assignment? Well, we'll see. Um So, So, is kind of the dilemma. Um If If I have 16 computations and and eight sort of uh threads, you might say, how do they how do they fight? Well, they can try and be You can You can assume that all the computations are fair

If everyone If all of the work was doing some very short amount of work we could be like, "Okay, I don't have to worry about them." You know, it's going to be it's sort of non-deterministic who wins, but they're only going to use that thread or that CPU for a very short time. So it doesn't really matter. The problem comes when, you know, it's like you're you know, I I I I my kids that they're they're they're fighting over the Legos. There's only so many Legos. They have to fight over them. So uh what if what if what if nobody, you know, gives that CPU back? What if somebody's hogging a thread? Well, now there's less threads available and we can't do as much work. So that that's sort of the normal problem. So we need to start thinking about um how do we share this resource? And one concept that we can use um is is the idea of an asynchronous boundary

Uh so here we go. So So we have these IOs We have these effects. We don't know what they are. We don't know what their code is, right? We've We've wrapped up something in a in a box. We said, "This is an IO and I don't know what it does, but eventually it's going to return a string or eventually it's going to return a unit." I don't know. And maybe that IO is made up of other IOs sequenced together and you know, in a for comprehension. Or maybe that IO is actually uh the par traversal of a bunch of other stuff and there's a gazillion little IOs in there. How do they all um how do they all actually get scheduled so it's a bit more fair? Uh the idea in in in Cats effect and in other um you know, sort of foundational libraries is that somewhere there's going to be this boundary um and in what in a particular one that you could perhaps call yourself is this shift method

And it basically says uh it's like a you can think about it as like a I tend to think about it as a hint. It's a hint to the runtime. To the outside world shift doesn't do anything. It just returns unit. But to the runtime that actually is running the effects, it says, "Hey, um you might want to let somebody else do some work." Um and it sort of uh one effect that it has here is that it sort of reschedules the rest of the computation to run on some other thread. So, if there were a whole bunch of work that somebody did a par traverse or a par map and there's a bazillion IOs, they're all fighting. Um these little um asynchronous boundaries are a way that sort of uh makes everybody all allows everyone to make progress. It says, "Ah, maybe you need to take a little break." Uh the rest you know, the rest of whatever you're doing will happen

It just sort of goes back to the end of the line. And that gives all the other work a chance to to to to continue and make progress. So, it's a bit of um in some senses it's a bit of an implementation detail, but it's a really important one because if you were programming some sort of loop that is doing all these things, you might want to say, "Okay, now's a good time. It's not so critical. Let somebody else use the underlying resources and I'll get back to what I'm doing." Uh if you ever have a you know, you can think about how you would make a loop in the world of IO, it could be really aggressive. If you just put a while loop uh if you constructed an IO with a while loop, it's just going to eat the CPU. Um so you need to be able to say, "Whoa, I'm I'm cool, you know, I'll I'll come back later." You know, where later is like, you know, a millisecond, so don't worry about it. So, when we have lots of things that uh when we have this abstraction of the effect and we have so many we can now sort of declare that they're running in parallel, we have to have a a notion of we want our our apps to make progress

Um and sometimes uh sometimes we need to sort of categorize what our IOs are doing. Um we might have a a very um CPU expensive IO uh that we could use the idea of an asynchronous boundary to sort of say, "Okay, I've reached a good point. Maybe let somebody else have a turn. You know, please share with your sister." Um but there are other kinds of calls. Um the dreaded blocking. You know, what if you Blocking is so much slower that you know, it's so much slower we give it a new name. We're not just computing the digits of pi, um we're talking to the file system. We're talking to uh somebody over in the next continent, you know, a network packet happens

We're waiting for a response. Those are much much slower. So, if I had an IO that represents some blocking thing that is a a thousand or a million or a billion times slower than some CPU operation, I I I want to say I want to like put it off somewhere else so it my application who might be doing something more important can use my underlying resources of my threads of my CPUs. Um so, luckily Daniel's here. I stole this quote from him. Uh it actually he actually said it. So, uh, how do you know if something blocks? And a good heuristic is if something doesn't have a callback API, then you know it's blocking. So, if you see something that does have a callback API, so callback API would be like on complete, on error, you know, that means you're registering something for something that's, uh, you know, you're going to get stuck handling

So, that's one heuristic. Um, so if you have something blocking, you're reading from the file system, you're making a network call, you're talking to the OS, you're talking to the Java API, we need a different strategy. Uh, and in Cats Effect 2, this changes in Cats Effect 3, so put on your Cats Effect 2 goggles. Um, there's just a bit more, uh, of an abstraction. So, instead of creating just, um, any old IO, uh, we we sort of decorate that IO using a block what's called a blocker. And we basically say, we sort of mark, we we give it a an effect as an argument, and the blocker says, "Oh, this the IO that we're giving it is really going to is doing something blocking." And it's sort of, uh, you know, we can we can look at the code and say, "Aha, whatever the argument is to the blocker, that thing is going to be blocking." So, uh, it gets marked as special and it's going to be running on a different set of threads. That way, the thing those those IOs that are not doing blocking work can continue. They can still be like, "Aha, I have access to the thread

Yes, yes, I can do my work." And only when, you know, when that blocking, um, when that separate pool of blocking threads gets woken up, then, uh, work can then get continued back onto the main thread pool where everything else runs. So, here's a little Here's a little program which demonstrates kind of how blocker works. And uh we have a regular IO that sort of is, you know, non-blocking or, you know, is doesn't involve blocking is probably the better way to say it. We sort of mark this other effect as, "Oh, this is this is going to do some blocking work. We're calling a Java Java API. Beware." We're going to We're going to push it over somewhere else. And then, you know, some other effects that's going to run after the blocking effect finishes. So, we see that uh we're on this What thread are we on? We're on the IO app uh IO app's, you know, default thing

And then, we can see that the blocker is on a completely separate thread. These are managed um totally separate. So, now blocking work does not interfere with regular good old uh IO CPU-oriented work. And then, after the fact, uh we're back. So, you'll see these uh you know, in your in your code, in your libraries, you'll see these blockers being passed around in Cats Effect 2. In Cats Effect 3, um there's essentially a constructor um for creating block blocking IO. So, it would be like IO.blocking, and you you declare at that very initial point that whatever the the um parameter is, I am blocking. You don't have to have a separate blocker, which is nice

There's less concepts involved. So, now there's sort of two two big contexts in the world. um There you know, there's there's different ways to to slice it. We We We We We can launch all this all this stuff. We have parallelism now. Once you start to take advantage of that parallelism, there's sort of different uh flavors of work. There's uh and the dangerous flavor um that you need to be aware of are those those blocking tasks. Things that don't take them, you know, nanoseconds, but take milliseconds or even second, you know, seconds to complete

We want to push those into a different into a different sort of section of the world, and we do that via this blocking context. And then sort of behind the scenes to ensure that the work that we have scheduled makes progress, there's this idea of an async boundary. They happen automatically a lot in Cats Effect, so you don't have to explicitly put them in, but we can put them into our computations, into our declarations of effects, so that they can that they can proceed. All right, so that's that's context. Uh we had parallelism. We have some, you know, you get it if you have so much parallelism, you got to start partitioning your work. Um sort of zooming out a little bit more, um more at the level of how do we organize our applications? Uh I'm going to talk briefly about managing resources, and then and then I'm going to finish up. Um so so what what's a resource? Um so and so I I sort of I ask the question um to sort of motivate the um there's a there's a corresponding abstraction inside of Cats Effect

Um so the idea of a resource is that um what we can think about is something that is that has some state perhaps, and it's, you know, it's somewhat expensive. We we want to keep it around. So usually the I would say the the classic example of a resource would be like a file handle. So when we deal with files, uh the usual API is we would open the file. Um that would sort of produce some sort of handle or file object. We then use it, you know, we read to it or write read from it or read write to it or and so on. And then when we're we're done with it, we have to close it. So, you know, this is especially important if you're writing to it

Uh once you close the file, you can't write to it anymore. Or if you try to, there it should be an error. Um if you, you know, in in the old days you could actually it's still pretty easy, but you might forget to close the file. So, maybe you're just when you're writing, you're actually just buffering things into memory and not, you know, when you close the file, you're sort of that final commit ensures that those bytes get written to whatever IO interface. Um so, a resource is sort of the idea of uh well, not only are we trying to do something like write to a file, but there's some state that we need to manage. There's there's a step that sort of says, "Okay, uh this is how I acquire my resource like a file handle. I can then use it. And then finally, I have to release that resource

You know, just like memory, I have you know, I I acquire a block of memory, I use it, I do whatever I want with it, and then I free it. Um that pattern happens over and over again. Um so, in Cats Effect, we have an abstraction for this. And and uh so, it's called a resource. And the way you build one, so the resource the the job of the resource is to sort of separate that um acquire the acquire and the release step uh sort of capture those right now. I know how to create a file handle. And I know, you know, how I know how to produce a file handle, and I know how to close the file. Uh and I'm going to capture that logic into into a different kind of abstraction, and we're going to call it resource

So, so a thing to notice here is it takes two effects as arguments. I'm I'm to have an effect which produces, you know, sort of the underlying resources. Produces an A. A would be like my file. And release is an a is a function which produces an effect. It takes the the resource, my file, and then and then produces some effect. So, this would be be the the closing of it. And then you get a resource out

So, I'm not I'm not in IO anymore. I get a resource. Um So, once you So, here's a here's a very toy example. Um I'll start here. I have a I have a I have a very important string. This is my This is my resource. And it's involved in IO. So, I I when I acquire the string, I'm just going to produce some string

And I print some little debug information before I end. So, I'm acquiring it. And when I when I am going to when I'm releasing my string, I just say, "Hey, I'm releasing it." Um so So, I captured the acquisition. I captured the release. Uh what can I do with a resource? The important part of a resource is using it. So, when you use a resource, you get the the acquired value. You give it a function which produces some effect. So, in this case, I get a value of type string because my resource is managing a string

I get my string and I can do whatever I want with it. But once my once my the effect that I returned, when I use it, I my job is to produce effect with the with the value of the resource with the resource. Once my Once my effect is complete, then, you know, essentially here, after this effect runs, uh that's when release happens. So, you get this bracketing effect. We So, in in sort of the debug output, I acquire it. It gets used and then it gets released. So, that that pattern, you know, we have we have it all over the place. So, so you can imagine creating resources where the resource value is a file is a database connection, you know, you acquire database connection, you use it, you run it with some you know, SQL or whatever, and then you're done with it and the the the connection goes back into the pool or it gets freed or whatever

All those patterns that um are a lot more implicit in your code become much more explicit. You make a resource and define how you get it and clean it up. And somewhere else, usually, you know, often in the top level of your application is where you you have resource or maybe, you know, often have more than one and you use them. And this is like your business logic. This is is a dependency. This is dependency management in Cats Effect. I have a dependency. I say how to make it

I say how to clean it up if it needs cleaning up. It's someone else's job to use that dependency. But it's guaranteed that when you use it, it gets properly created, used, and then released in the proper order. So, that's it's very handy for all sorts of things, both ephemerally and sort of at its top level of your program. You define your database connections, your you know, anything that sort of need needs set up and tear down. You can always model as a resource. It's super handy. Um I didn't have time to do all these things, but you know, here's some here's a couple examples

We did we talked about file handle. Uh You uh there's a really cool method you should check out on iOS called background. Uh it essentially would fork your IO to run, you know, run on its own. You sort of you sort of detach from it, but it's running, but you get a resource that represents um the lifetime of that um of that fiber, of that effect. So, say you had an effect that was just sort of keeps on doing stuff, a clock, uh uh uh a sort of sort of reactive loop. Uh you could then shut down the the resource represents the lifetime of that um background task. Uh another thing to note is we can compose resources. So, I just had an example of one resource, but if you had two resources, you can uh you can map resources, you can flat map resources, you can map n resources, but you can also um par map n or par traverse

You can do you know, you can do you can par essentially par map n resources. So, that would acquire resources in parallel, and then their release would happen in parallel also. So, it's a super handy um structure that's intimately involved with IO and effects that captures a very common pattern. Okay. So, that's uh that's all I have. There's really a lot more. Um let me let me go quickly summarize. Um So, we started out talking about effects, and I I wanted to really emphasize again because we should all, you know, have this internalized

Like, substitution is super important because it gives us so much. It's the gift that keeps on giving. It uh if we delay the evalua- the execution of these side effects, and and if composition allows us to uh be referentially transparent, we can build up our programs and refactor them and and nothing will change. We won't introduce any bugs by refactoring. That is a great, uh, a great thing that we have access to. So, if we want that, uh, we can use Cats Effect IO. We capture these side effects that are dangerous that don't support substitution. They get this nice safe interface

We compose, we do maps, we do flat maps, we do map ends. Uh, and only in the beginning uh, of our programs do we say unsafe run. Or we use we even use IO apps. You do not even have to say that. We just declare what our program should do and it's the system's job to actually do it for us. Uh, to to, you know, we we're not responsible for those side effects. We've properly wrapped them up. Uh, we get parallel execution

We sort of sprinkle our pars in. We can par map, par traverse, par sequence, par tuple. Forgot about that one. That's a good one. Uh, now we can run many things that are independent and we can take advantage of them, uh, take advantage of the resources that we have on these big fat machines that we have. We can use them for their you know, what they really can can do for us. But, once we start trying to do more and more things in parallel, we're going to run into the actual, um, limitations of how many CPUs we have, how many threads we have. Uh, if we have threads, we don't want to block on them because we could be interrupting other people's work

So, uh, there are a couple concepts involved there. We have this asynchronous boundary which says, "Okay, I've done enough work. Maybe it's time for someone else to do the work." And also we have sort of the notion of say of categorizing our effects as blocking or or non-blocking. And those will, um, use separate set of resources so that so that everything keeps chugging along and we don't sort of, um, starve ourselves. Uh, we There's a very common pattern of acquire, release, and acquire, use, and release. And sort of the the resource data type in Cats Effect lets us, uh, encapsulate that. Very, uh, uh, useful for dependency lifecycle management. Uh, and there's like so much more, um, there's a notion of fibers which I didn't get to, async, other ways of building asynchronous computations

There's a whole bunch of concurrency primitives, um, for sharing state, for, um, doing semantic blocking, you know, you'll you'll get to learn what that is, um, later. Um, but that's, I I think a good amount to, um, throw at you there. Hopefully, um, hopefully this makes you excited if you haven't seen it before, it makes you excited to, um, change little by little, uh, your your applications, your business logic, the libraries you use. There's a huge ecosystem. Um, uh, here's a here's a super brief list of places you can go next. There's always the the Cats Effect website. Um, I learned all of the tricks from these great folks who are online, uh, especially this Gitter room. Um, you know, you have you have a question, people are going to help you

They'd love to help you. Um, uh, and then, you know, I I I I saw that people had the same lots of questions about, um, Cats Effect. So, uh, and, you know, there's some great blog posts out there. I And so, my my current project has been to to produce this book. So, it's available now for early access. You can, uh, you can check it out, essentialeffects.dev. Um, I think it's a reasonable price for what you get. It's like 170 pages

Let me know if you have any questions about this material. I'll post my slides. You know, ping me on Twitter anytime. I'm A Rosine. You can hire us Inner Product. We're We're If you need help migrating Cats Effect, you know, no problem. Training your staff. Whatever kind of side projects, we we can help you out

So, I'll leave it at that. If there's any questions and uh and hopefully there's also some some the deep experts are also online to answer any extra special questions that I may feel uh uh too tired to ex- to to answer. Great. Thank you, Adam, for a great talk. Any questions anyone on Twitch, anywhere, on here on Zoom? There's been a few questions on um on Twitch, but I think Daniel's handled those other than us someone asking for a question related to comparing uh Cats Effect 3 and ZIO. Um I don't know Daniel, but do you want to expand on it? Uh today or Yeah, I can I I can expand on it, but um if there's interest in it. Um I I think the So, the the the TLDR for me on on comparing ZIO and Cats Effect, so it's difficult to compare them together one-to-one. Um Cats Effect is is a library

Um Cats is is a library. Um ZIO is a framework. And I I use those words to kind of evoke the the like kind of like old like Spring versus Juice idea. Like, you know, some of the stuff that like those of us who've been unfortunately around in the Java ecosystem, you know, from from those days might remember some of that. Another one you might think about as a metaphor here is like if you think about play versus like you know, like you know, like a Sinatra style like library for you know, doing rest routes. That that type of distinction is what it is. Now, again, there's no there's no disparagement in that comparison, right? Like Play is a fine framework. Spring is a fine framework

People write applications in Spring today and they choose to do that for a good reason. And the reason that you choose to write in terms of a prescriptive framework is the choices are made for you. When you want to do something, there's there's a a right way that it will sort of guide you down. If you want to do dependency injection, you're going to use Z layer. If you want to do sort of, you know, error handling, you're going to do it in this way. Like you know, all all those sorts of things, um Z layer has answers for it and will sort of guide you in that direction. And especially for beginners, that can be a very appealing path. The type-level ecosystem, conversely, is is designed to be a composable set of libraries

So, what that means is that it's it can be at times a little bit less batteries included. Um like you know, you you're sort of have to make some choices as to, you know, how you want to compose things. And that can be a bit daunting. I think there's there's ways that that has to be improved and I you know, I definitely I feel like we can own that. But the advantage to it is that you will never hit a point where the library can't go with you any further. Like the classic problem with frameworks is like as long as you're in a happy path of the framework, the the cases that they considered really well, the frameworks could be awesome and you're going to have a like a magnificent experience. It's going to be just completely unparalleled and really really nice. And then there'll be a moment where you hit a wall and you're you're just a little bit outside what the framework your problem space is just a little bit outside what the framework considered

And you can't really do anything. Like you're sort of stuck and you have to fight the framework from that point forward. Um this is a problem with Spring. It's a problem with Play. It's a problem with Akka sometimes and it's it's definitely a problem with Z layer. Like you will hit that wall at some point. And you might hit it in different ways, right? Like a classic example is like, you know, notches, doing just you know, doing tracing, distributed tracing, and threading spans to your application. Um, you know, there notches is a great library for doing that, but composing it with ZIO, while you can do it, it's a little bit weird

Like it's kind of like this thing off on the side. It doesn't use the normal environment. Um, you know, there's there's other more elaborate examples that you can come up with. Um, whereas like in the Typelevel ecosystem, all of those examples just sort of layer on compositionally. So, I think of it in kind of the old, uh, you know, Eric S. Raymond metaphor of the cathedral versus the bazaar, right? Like the ZIO ecosystem is very much a cathedral. It's very curated, sort of like, you know, this set. Everything kind of fits together in exactly this prescribed way

Um, the Typelevel ecosystem is a little bit more like a bazaar. You can kind of mix and match. Um, things compose together. You can build arbitrarily complicated things. Um, but it can be a little difficult to find your way. So, I see the best way to kind of bridge the gap between these and provide a really compelling experience, uh, for newcomers and experienced users alike, um, obviously within the Typelevel ecosystem, cuz that's, you know, where I'm biased. Um, is like, okay, I want to give you the bazaar, so that you have choices, but I want to give you a nice map to that bazaar. I want the bazaar to be really well organized

And I want you to have a really good experience. When you walk into that bazaar, I want you to know, you know, okay, you're looking for shoes. Here, here's some really nice shoes. Like the metaphor is getting a bit stretched.