scala.bythebay.io: David Cleaver, Finding the Free Way
Recording: scala.bythebay.io: David Cleaver, Finding the Free Way
I you alright great let me make sure I've got that going all right so if you came hoping to be taught free that's not really the focus here it's more like our experience story Kelly had a great talk on Friday if you missed it I really advised video so a little quick background I've been about seven years working for comcast about seven years playing with Scala I'm a technical lead on our team so I have the distinct pleasure being able to bring in new techniques and champion new approaches within team and that includes Scala which I've been championing for probably about that same seven years and working full-time in about two at comcast so so we develop platforms and we had to develop a portal for our platform so that it could be self service because platforms stink if you need a lot of help and tickets and things to to get started and get going we think within comcast which a lot of things do so we were in a spot where we were developing code we had code that was hard to understand very unpleasant to change and the team was very frustrated and unhappy and it was it was like Lord of the Flies kind of situation in our team so we said hey what about the free monad and actually the team brought this to me it was a technique I was already interested in so I said yeah let's go let's let's pursue the free monad as an approach to do this we can you know capture the structure of our computation we can interpret it later we can have the different interpretations for testing and production possibly analysis it'd be really useful including you know we could use cats will build our simple data types and we'll describe our algebras will combine them with co-product will build simple interpreters and we'll combine those interpreters it's easy right like no problem let's just go so we watched a lot of talks i watch true NARS talk Raoul Raja has the functional programming patterns version 3 I highly recommend as an approach to this the big problems that i would say that exist are that a lot of the talks are sort of toy examples you can't do real examples in a talk honestly but it's sort of hard to you know you have to take those examples and sort of translate them into the reality of getting your job done so what did we do to get through this at first we sort of had everybody run off and try to learn free on their own that didn't work and we had you know maybe two or three different approaches to to writing the code and developing everything and so I said okay we got to stop this is this isn't going to work like this we got together in a room and we started from scratch and I took the very first end point that we wanted to implement and I said what do we want to see here for in our case it was like getting all accounts and so we said okay let's write an algebra that that's all it does and then we slowly added to that and for that first thing we built the very simple interpreter that was purely in memory didn't do anything real and I continued that approach as we built up algebras and you know everything synchronism xin memory they were all mock great things we could use for testing but they didn't still didn't do anything so i'm still getting kind of sideways glances from the team like hey you realize you're not doing anything yet but once we were ready once we had enough momentum built up around the algebras i said oh you want to do something real let's take one of these and make a real interpreter go hit the datasource go pull out the data and return it through and and that's I think when the lights really turned on for everybody because they wouldn't suddenly went oh you just went from being having nothing having nothing real to having something real in you know a few lines of code one approach we we use play within our team one approach that we did we created a custom play action which applies the interpreter one of the first things we noticed was if every action is going to be running free they're all going to be running that same code that map's the interpreter over the free probably want to hide that away make it look cleaner so our play actions literally look like they just take free for comprehension and build up free structures and all of the common interpreting all happens in one place which is really nice so all the code became much cleaner and easier to look at so how did we design an algebra would probably be an actually better question so you want to consider exactly the code that you want to see right what are the inputs what are the outputs what are the complexities that should or could be hidden that's an actually important question a lot of my team wanted to be like well this this is something we should be concerned about it should be in the algebra and and I had to kind of say well you know that's something that yes it's a concern but it's one we can push off to the interpreters and not worry about it all in our in our controller and concurrent modifications was was the big sticking point there you know they said oh we have to expose the fact that we're doing it updates that we can do kaz values and all kinds of stuff and I said well you know first of all there's like three people hitting this at a time probably not as important let's definitely push it off I'd much rather see you say get something make a modification put it that's what I want to see in the controller versus an explicit say update step so we also hit away things like access to the play request way uh and and also you know we can hide things like retries if we want to do retries on an interpreter we can kind of hide that away from the from our business logic which was useful so designing an interpreter right all of your interpreters with free have to produce the same output monad this doesn't mean that you have to build all of your interpreters to produce the same output monad as it turns out we decide we made very good use of the functional abstractions available in cats we applied monad error where possible if an interpreter didn't care what monad it was in and all it cared about was signaling errors we went we went to monett error we hid the the play request in Cleese Lee and we very explicitly said hey most of the interpreters don't care about the play request so let's lift them up two Klees Lee rather than having them worry about the full details this made our interpreter design a lot simpler and it made combining them later easier also consider you know so consider Cleese Lee for things that you want to hide away and pass through to every interpreter and state t4 things where you want to store state so our calves problem I approached you know my developers and I said hey we can solve this with state T and we can solve it in the interpreter and most of our interpreters won't care and only the one that deals with kaz values is actually going to care about that so yeah so the lifting transformations that I mentioned they became very important right we lifted two Klees Lee we lifted you could lift from Cleese Lee to state we could build lots of those different kinds of lifting interpreters and that made each individual interpreter much simpler for the developers to grasp and then later combined them into more complicated structures and then also our individual interpreters are protected from changes to their output we want to change our output monad most of the interpreters aren't actually going to care we started building you know oh we have an interpreter decks or we have an interpreter future we ended up with hey there's one interpreter to monett error and it covers both of those cases very cleanly for us so what do you need we used cats I highly recommend it it provides all of the you know base free support all of the useful functional abstractions that we used the kind projector is critically important for a lot of this stuff I find being able to write your types in a clean way and not have to do type lambdas I understand type lambdas most of my team does not understand type lambdas so it was very if even when we were using them we had to kind of hide I had to kind of hide them away a little bit so they wouldn't worry about them so and then also miles is si 2712 I put in so that we would write a lot less types than we were writing previously you know now just upgrade your Scala version and you should be good you know more tools would be good I was pointed towards freestyle to last night and I looked at it today and I'm very excited to try to bring that to my team if you're just starting out there's another library called free k which is interesting we were very far down the cat's co-product path and using free k would have involved a lot of changes to the code and so we didn't go that approach but I did build some other tools that we use to sort of fit things together in a clean way and reduced boilerplate so a simple example of one of an interpreter which oh yeah what does that look like yeah there we go simple example of an interpreter this is you know I mentioned retrying errors so this is an interpreter we could build wrap it around another interpreter and also provided a function that says hey given an error how many times would you like to retry that and it's built right on monette error so that we can be fairly flexible here so we take in the input algebra and we need to produce our output monad and so you know i wrote the simple attempt attempt is going to say you know translate translate the interpreter here use the interpreter to translate and then handle the error by checking and if you know if your attempts is less than the number of times you're supposed to retry attempt again on otherwise raise the error very simple and then you just recall a temp 0 it's just it's just that easy and we could wrap it around you know calls to our data source calls to back-end services things like that and so these become reusable tools which is really nice realization for us let's see so the other I mentioned lifting interpreters so you know I wanted to be able to write this interpreter and then lift acolyte two Klees Lee right so we created an object to hold these transformation these these lifts and it's very simple what we want to do is we want to just take any output and turn it into pleasley so that we so to do that is very simple ignore the argument and return exactly what you have as your input so are we done so he's a good question our code has become a lot cleaner and easier to understand it's easy and pleasant to add new endpoints I've got happy developers that were gushing about the techniques to my manager which is always a plus but we're not done I think I'm always striving for better cleaner code organization less less Euler plate in the code and we're still learning how to sort of build the power into our interpreters and then figure out what else we can do and additionally you know we need to spread the knowledge spread the knowledge outside of our small team and into into larger teams within comcast to that end we did on board a new developer who was fairly even new to scala and she picked up very quickly the techniques that we were using and so I consider that a win for our education forward and using this technique so I guess sort to conclude maybe a little early you know write the code that you want to see and then make it work this is the mantra that I've started using it work and this is actually really important because I had a lot of developers that were writing code that worked but it wasn't code that you wanted to see you know the free Monette is one tool to do that which requires some initial ramp ramp up its it's not easy but it is ultimately rewarding my advice is to find an approach that works for your team and then share so you can always tweet me at at Diaz cleaver and that's it [Applause] mhm so are we considering co free essentially as an analog to free I i watch John's talk and I'm very excited to learn more as i said i bring techniques in right so i'm going to explore it probably first and figure out where we could apply it i have looked at FS 2 and i'm very excited to try that as well and get that maybe into our team so i have the advantage that Michael works for comcast and so I've already told him that I'm going to co-opt him into you know some training because I can convince my boss hey we just by Michael lunch he might he might do some training for us so so yeah I think it's a technique i'm very interested in so any other questions yeah I've done some what is sorry what is the overhead of free monads I've done some sort of back of the head you know not even writing it down on a napkin type calculations I feel like there's not maybe as much overhead as people say that there will be there's definitely going to be extra stuff in memory but a lot of it's going to be eating space right so it's going to be stuff the JVM is just going to throw away as soon as it gets an opportunity to so we haven't seen a lot of performance impacts on it so you know what I'll evaluate it performance is a funny thing right people talk about making things faster but they don't really talk about where they need to make things faster and so I think you have to evaluate in the case you know I used to do tight loop optimization things right and at that time in the JVM you couldn't even create objects without it being slow so we we eliminated all object creation but for something that's you know a restful call that may be a customer makes every five minutes if it's an extra couple of milliseconds slower I'm not really going to lose any sleep over it so anybody else oh thank you very much [Applause] you