scala.bythebay.io: Scott Maher, Using CoProducts to stitch together algebras for scalable complexity
you like that guy told you i'm scott mhairi I'm from twilio and the tech lead on the messaging team how many of you guys read the actual description of my talk on shed stead whatever I that was a long description there's no way I'm gonna be able to talk about any of well all of that it took me at least 20 minutes to read everything that I wrote so I apologize if you expected me to put all that in there how many free monad tutorials have you been to at this conference you got room for one more yeah alright cool I don't know if it's the free monad or free monads how many people use it regularly or half okay cool I guess assume that you have a little bit of familiarity with the free monad I won't go into it in depth the basic stuff the tutorial stuff that you see online basically the question that I've been trying to answer for myself is is this really a reasonable way to build programs should i use h i build services out of it is this practical or is it kind of more academic and the answer is sure why not yeah it actually is really really useful for especially for programs that do a lot of Io because I o tends to unnecessarily confuse your business logic so if you got one program that's mostly pure and then writes the output to a file at the end I don't see a lot of utility in the free monad but if you're building say a service in a service-oriented architecture maybe you're talking to a database you're doing like varied in multiple different types of RPC calls maybe queues are involved then this is really useful because when you're writing your tests you don't really have to mock out that type of Io you can just write a testing interpreter for these actions and defer execution to another time or different implementations so in the classic free mon at examples that you see around there a little bit contrived you would write your dsl your grammar which has your change that you would execute at another time in this example you've got a store with get put and delete you see this before you write some kind of a program that ultimately returns a free of store and whatever kind of result type that will eventually get interpreted by your interpreter it kind of looks like this if your database lives entirely in memory and this is problematic because you don't have a way of combining that dsl that storage DSO with other dsl's with other grammars that you might actually use when realistically building complicated programs right and the whole point of the free monad is to make your program simpler you can't break that into libraries if you add more and more terms your grammar does more things or mop your interpreters guy get more bigger monolithic if you split your company into multiple teams or if you work for a large organization eventually going to want different people to be working on different parts of the grammar or focus on different domains so you really need a way of not just building grammars and dsl's and that are expressed in terms of the free monad but you need a way to sort of combine multiple different grammars together so that's what the co products all about you can see here it's a type that you get from a lot of places if you use catch you can get from that if you scholars said you can get from that shapeless has an implementation and it's a way of saying it's like a higher kind it either really it's a way of saying this thing could either be an F which takes one type argument or it could be a G which takes another type argument that can't be both and the way you would apply it to your free monad program is you'd say this this this step in the program is either a this domain or that do menor maybe it's this other thing and you can use it for all kinds of different reasons it's kind of you the cat's vernacular it's like an either k because it's got that higher kind so I'll take you through an example looking at blog post examples of code looking at Tech Talk examples of code feels a little bit like flipping through a gallery of stock photos you have this very general sort of case that you hope relates to somebody in some way so I know this feels a little bit contrived but it kind of makes sense maybe your company is asking you what's building an art store so you've got a domain of yet art these are domain objects that you're selling and you also maybe sell frames and here's your grammar you've got this inventory dsl which allows a user to search on artists and returns a list of art and you can also query the query the inventory for a list of frames are available for sale and maybe another team is working on more of the shopping cart functionality or maybe this is something that you buy off the shelf from a third party and you've got a grammar you've gotten out dobrze to describe all the things that you would normally do with a shopping cart leg add item sit car and list your carry arms and maybe make a new card because you'd abandon the old one and then lastly you've got some sort of e-commerce provider potentially the e stands for electronic in you're using stripe or paypal or Braintree or something and they give you a way to purchase items so this is your grammar for talking about that which has only got one member takes a car tidy sum payment info apparently they only accept bitcoins and will give you back a receipt of some sort so co proc was a little ways ago I just a reminder it's this thing that takes an F for g one or the other and you would use it for your art store to combine together all these different grammars and since it's only got two sides you would first build up things one ahead to a time so since you've got three total grammars so you want to combine together you start by combining the inventory of the cart and then you would take that you call that searching cart or something similar and then you'd tech onto that the commerce power which is your payment provider and it would give you a type called rat store because you're bad at typing and you bought the domain and mouse just called rats or so and just to be super clear this inventory this cart this commerce these are all the types that represents your free grammars these are the free of your abstract data type algebraic data type so will then create our smart constructors and typically in a lot examples you see online these are just aliases for free lift see to lift your grammar into the free type in this case we're doing something a little bit different notice this class inventories its parameterised on f and it takes an implicit of inject at the top so first of all you can see that these smart constructors you have to instantiate them which is kind of interesting and the f in this case is supposed to represent another grammar that you want to work with so for us this might be a shopping cart but it's not dictated it's generic on f whatever that might be and you also expect a instance of inject to be provided to you and we'll get to that shortly and then the other thing to note here is that we use free inject into a free lift see and that is basically a way of using that type class that's passed down if you are expecting your grammars to be used with other grammars you kinda have to do this unless you have some other scheme for composing your grammars together so it's kind of a planning ahead type thing so the inject type class its I don't see love used for it other than this situation but I have and looked at in depth maybe somebody can tell me more but ensure it's basically a way of taking an F and making a G in some cases with the inject method and sometimes maybe you can get an F out of a G and that's what it's talking about so there's a few notable instances of inject that you get for free out of the cat's library probably the same is true of the scholars ad free library as well and other implementations of this because it's such a useful and common pattern this basically allows you this cat's free left inject which is pre-baked in tow system helps you build up these co-products grammars from existing from basically existing shapes up here so it automatically allows you to go from an F to a co-product of F and G which in this case is our F might be like our commerce grammar and our G is our combined commerce and our combined commerce and shopping grammar or combined inventory and shopping grammar however you sort of want to put these together so kind of at the end of this we then give ourselves a handy constructor here which for actually can like I said in the beginning we need to actually build an instance of our smart injectors and this is our constructor for that with all the implicit defined here so say hey I want to give you a instance of inventories but you got to tell me what you want to mix me with and it can be reflexive it could be I'm going to mix myself with myself in case you don't have any other grammars that you want to mix us in with makes it kind of sound like the cake pattern huh here's the implicit of that and then I'll give you a new inventories so here's our dsl again this combined from these two different type are these three different types and then what's really really cool is you just call this say I want in inventories to be injected into rats tour co-product implicit it's free in place it's just sort of know how to build the structure up with the co product to actually inject inventories into that and its really really neat it's kind of beautiful how that works so now it's time to actually write your program in this case I'm going to build all my different grammars up here the cart the inventory check out I'm gonna buy art for myself so this type ends up being a rat store that ultimately gives me a receipt I get some frames I search for artists Scott mayor I got lots of portraits like to look at myself get a new card add my art to the car add a frame to a cart and then check out and purchase yielding my receipt can you spot the unsafe operations in this yeah there's two heads you just totally ruin everything all this effort to be functional I look what you've done this is kind of what you expect right where this you have one domain the one interpreter kind of like as if you're just working in one what with one domain that time which is nice right in the previous slides we talked about how you kinda if you're building these grammars you kinda need to expect that it's going to be injected into other grammars here you have a check out interpreter I would never do this don't don't do this that the target monad is it goes from commerce to ID [Music] so you would do all your normal side effect ii type stuff here maybe and then here's where we actually do the interpretations so we're taking a command to pipe commerce there's only one type of command here which is purchase but if you had different interpreter like for the shopping cart you'd have like an at the car list items and cart new cart different cases and then you do the business logic here which apparently I decide to write this as a sinks I thers so I gotta wait for the result definitely would not do that and speaking of that in the real world instead of having an object check out interpreter you'd probably have something that would give you some kind of dependency injection from it so you have a class check out interpreter and maybe it takes a strike client that's a Java client under the hood or something like that maybe some sort of thread pool and you would probably go from commerce which is your grammar to I to something that's not ID this is a good choice it might just go directly from either from commerce to either T of future where your error domains kind of represented on the left side and your your desired result which would be receipts on the right side that's a normal thing you might mix in you you might build into your monad stack a writer t you do all kinds of stuff IDs not a realistic choice here putting your interpreter together can be a little tricky so that's just one interpreter that was the Commerce interpreter but you got many more and to actually run your program your guy need to combine them all together right so here's a co-product I think this is the last time so if you didn't get before and we want a function that looks kind of like this from rats door to ID because that's why all of our interpreters are going to in rats door is really expands to this Co product which kind of looks like a binary tree right yeah Commerce and then at this layer on the right node you got Co product again which goes down to an inventory or a cart grammar so our interpreters are function case which are another way of saying their functions that operate on the higher kind of HMG which are type constructors which take one argument kind of like Oh product is on F and G where F and G are type constructors that take one argument function K is kind of like that argument analog so you get this apply it takes an F and gives you back at G and we also have a handy function on function K which I probably should just moved within these curly braces called or and what or gives you is it basically takes your function K and other function K and it turns it into something that takes as an argument a co-product which is kind of exactly what we were building up a couple of slides ago right with this is actually what our grammar looks like it's a co-product of law of multiple different types of grammars so we've kind of got away based on what our interpreters are which are function KS to combine them together now we just need to string them together in a way that string them together in a way that kind of matches this rat store structure here and it's really not that complicated just do things in the correct order and you should be fine so you start off with matching your inventory interpreter with your card interpreter and then you match that with your checkout interpreter and then you ultimately end up with a function K that looks the way you want to actually apply your free to it and now you can render your program all that work for this one line it's kind of anticlimactic isn't it but there you go neat I went take a few moments to talk about how neat that really is functions that return functions are really really cool we're kind of used to this as an idea now right as functional programming Scala people o camel Haskell whatever people right and really you can make the argument that all multi argument functions are really functions that take one argument to enter turn another function wxy to return izzy is really a function that turns aw that returns an X to Y to Z and so on and so forth you kind of leveled up with this you you built not just a program or a function that returns more functions your turn programs that return more programs because programs just like functions are just values and all the fries giving you is a structure that you can unpack step-by-step and evaluate step-by-step write a monad is a way of serializing things as steps and steps and this is a way of serializing steps into a data structure really he in yet you knows more functions so assume that your art store is doing really really well everybody's buying those self portraits of Scott mayor and you decide to buy the company down the street which happens to have a 3d printing factory or something like that and you said hey I'm gonna start selling 3d printed objects that people can buy through my website if you guys ever use like Shapeways or anything like that those are the 3d printing on demand services they really need so I'm not a domain expert in 3d printing but I assume this is what a algebra for 3d printing looks like maybe i extrude clanton these all look pretty reasonable to me and you can build these up and you can put these values anywhere because they are values right you don't have to interpret them anywhere if you don't want to you could actually have your other algebras return these program your other grammars return these programs somewhere so you know this is our inventory algebra again since we're adding new product lines you know we still find the artists or the arts we're still selling the frames but now we allow you to 3d print the Venus de Milo and you can choose how big you want it to be that's an integer so we only du meter increments so you can't get a half meter and since that function that a that step that that that part of your grammar returns a 3d printing program you can do all kinds of things with it you can interpret it right away you can figure out how to serialize it in some way push it on cue have somebody else hear it deal with it that kind of thing you know you can just forget about it you can do whatever you want and that's need you really gotta start thinking about these bigger structures as values these programs that you can interpret as values and do whatever you want with those values so it's a neat concept now getting back to I was talking about before it is the free monad or our free monads are is this a reasonable way to build programs and yes i think so i think especially in highly networked environments it's a very reasonable way because the payoff despite all that bullet or plate is that you can really reasonable reason about IO in a more controlled manner realistically your interpreters and grammars might have dependencies injected it kind of point that out in the Commerce example if you have stuff that does Davis I oh you might inject thread pools or like a slick client or something like that and again I mentioned earlier if I want to reiterate choose a good target monad i like to use either t for a lot of things because since i live in a networked environment most of the time you know i used futures a lot and i also don't want to rely on the built-in exception handling there but the mechanics of these free monads are terrible there's so much boilerplate why is there a co-product that makes me build a binary tree and all this boilerplate can totally be generated with macoris anyway i just want to say thank you to all of the people who have so hard to put so much documentation out there about this stuff the cat's tutorials not just on the free monad but on all the type classes and data structures are amazing thank you so much for the work on that Paul types on reddit Scala is really awesome and super friendly why the free monad is in free Kelly Robinson I think did that talk earlier too yesterday I missed it but it's also online it's really good and there's a ton of other resources out there so that's it for my talk thank you very much for coming [Applause]