Scale By The Bay 2018 Unconference: Joe Kachmar

Scale By The Bay 2018 Unconference: Joe Kachmar

Recording: Scale By The Bay 2018 Unconference: Joe Kachmar

named Joe catch Mar wanted to just kind of take the time here to walk through an example of a sort of way of mocking effects and creating sort of constrained effect flow interfaces and Haskell for applications that I tried out and am really not sure if it's a good idea or not so this is gonna be kind of a little bit of me showing code off and then a lot of me getting feedback from people to get better ideas to farm this out to sort of all of you so I'm gonna kind of dive right into it which kind of starts off by I guess I'll start with maybe a sort of problem statement here which is very frequently I find myself in writing applications not wanting to allow the full range of expressivity of say an i/o type or an effect type in neither Haskell or Scala and I can't want to say only give people access to interfaces that I know I have complete control over such that they can't do things like fire off an IO action that fires all the missiles or like you know deletes all the code on your computer it does something awful like that and so one of the ways that you can sort of do this in Haskell is by using the type class system to create sort of like an abstract interface for the effect you want to perform and so that's sort of what I'm doing here in this block of code I'm creating a type class called HTTP which is parameterized by some type M which is just only constrained to be a monad and that implements there that specifies a single method and the type class interface called call takes a URL a client environment and then returns something but this is sort of designed to be a very sort of abstract way of saying how do I make HTTP calls in the context of some application and then down here I create a function that is sort of the equivalent of what I would like an HTTP call to be with a name that says IO call HTTP implementation and this one has a much richer sort of set of constraints upon the types where I say I need to have some kind of I need to have access to some kind of context and I need to have within that context some kind of HTTP configuration record and then I need to be able to perform IO there and so this represents sort of a real-world implementation of this call method that's going to actually perform real effects and because I'm lazy I didn't implement it out I just sort of said this is how I would retrieve the manager the connection pool for my HTTP client connections and then undefined is just saying the telling the compiler make the types work out I'm not actually gonna write this function so for the sake of example this is pretty empty but the idea here is in this single module I can define everything I need to do to create the interface and maybe work with it so this is sort of that HTTP config type and it just has a single element within it called an HTTP manager that is the manager type which is the pool the connection pool for the HTTP clients kind of an interesting pattern I figured out when working with this is this sort of cool alright so now it's recording so anyway just a cute little trick maybe also I want to try and keep the ability to swap and move so like I want to try and keep the interface abstract on the way that the function actually needs to run relatively abstract so I can use it in different environments and so I'm using a library called generic lens which in Haskell allows me to generically derive optics for data types without having to do any kind of special extra work for it and so this is kind of neat because I can just create this type Elias has HTTP config which just specifies that the type context has some member within it that has the type HTTP config and then what I want to actually get it out I can use these lenses here which is basically saying in the contextual environment view something of type HTTP config and then within that view something of type manager and then that lets me get the manager out for my product type it's in the environment so yes so this is kind of what I'm what I'm doing here I'm saying in order for this thing to do what I wanted to do it has to be operating in a monadic environment that has Monad reader where the context in Monad reader the context type here is also constrained to provide the HTTP config type within it and in this sort of way I am building up what would I think maybe another languages be called sort of a dependency injection type pattern where I'm saying this function should be able to operate in the context of some structure that provides the environment it needs to actually do what it does but it doesn't need to know any more or any less than that and so that was sort of roughly what the HTTP clients rotation would look like maybe a bit more meaningful to people who've done this before is this is sort of what a logger would look like for it and I get the same sort of thing here where I say a logging scribe the thing that's actually going to handle pushing stuff out to the environment and doing maybe tracing is a type class that again implements monad and we implement monad for all of these because the only constraint this really needs is to be is to be able to work with Haskell's do notation and so then to do sequential steps so each of these is gonna need a monad constraint but not much else because we just want to use them in a do block and so we have all these methods that are just say take text and then em of unit so perform some effect and then do nothing else for all of these except for pop namespace which is just going to for some tracing namespace maybe remove the last talent that was pushed in there and again this is also sort of neat because when we do what I call the concrete implementations for these types we can defer entirely to the environment that we'll be operating in in the production instance of the application which here is using a logging library and Haskell called khat dip which does structured logging and here again I can say the concrete implementation for the info scribe is going to just be something with access to monad i/o so it can perform an i/o effect and something with access to what's called cut tip context which is a type class constraint from the logging library that gives it access to the ability to write out structured logs to its own internal state but this fulfills this interface here but for a production environment and so when we write any functions that want to take advantage of the scribe we're only going to use the scribe type class and then afterwards we'll wire it all up so that when we run the application and production the production version is going to use all of these real-world sort of implementations so the rest of these sort of scribe things are implemented very similarly I use the primitive functions from the library to do tracing error logging and then I've stubbed out the namespace stuff because I don't quite understand how I would put that in there yet and this is more for the sake of a fun little example and then similar trick to the generic lens stuff again here where I create a configuration record that's got everything that my structured logger needs to actually do stuff so the namespace that is maybe pushed into the trace the context and the logging environment that determines the severity that's logged out to the terminal and then I've got this cute little type class constraint that does exactly the same thing laws wheedle ends into that configuration record without actually manually defining any lenses or using any sort of macros in template haskell that would force me to do that and the logger one is even simpler because i'm saying that a logger is a subset of scribe and here you can sort of see how these constraints sort of in and relate the type class interface I have for a logger is almost identical to scribed because I want to say that a logger is almost a subset of a scribe if you're I'm just using a logging functionality it might not need to get access to the namespaces but here I can say I can defer the entire implementation of my logger to describe interface and so I'm saying that the implementation of a logger depends on some scribe and then each of the implementations of the the logger itself are just in terms of the scribes interface entirely and so you could sort of see how these abstract interfaces can actually be combined into each other in a way that you can build up more complex interfaces or stick to simpler interface depending what your needs are so that's sort of how all of this thing is all of this stuff is defined and as far as how it's actually used I have an example of an effect fulfill these interface would actually apply dear and so I want to perform some kind of HTTP call and so I've required that the M that I'm operating in is constrained the HTTP effect and I want to do some kind of logging call and so I constrain it by the logging effect and the actual business logic of the function is just that I am not admitting arbitrary i/o and my function if I uncomment this line lift i/o some of my tooling over here it will say you can't do that and so I have a high guarantee that the code that I'm writing can't do something outside of the constraints that I have imposed upon these interfaces well at the same time when I actually run this code it is going to be capable of doing things like HTTP calls and logging because the interfaces that I defined are is sort of admitting that and the way that this all works is with this sort of app new type and so if the type classes that I defined are the interfaces this type is the wiring and this type is basically how the application will actually be run and it shows how all of these interfaces actually get wired up to those implementation functions that I showed earlier and so an application in this case app is just a reader T around some configuration record that had that operates in the i/o monad and then returns some a when it's running so in this case I get to derive a whole bunch of stuff from Haskell here and you can sort of see how it has access to that monad reader context because I'm saying derive monad reader of config and so anything that operates within this environment will be able to use the monad reader methods for the config environment where I define the config record down here and so the config record is going to have access to the content config record I defined earlier and the HTTP config record to find in the HTTP interface file and so these are this is sort of that wiring that I was talking about and so here when in the cut tip functions I required a content context this is how I'm actually providing that cought type context to my app type and so things that execute within app are going to be able to use the content context constraint to get the information from the app type to perform all those actions and then in this case the scribe wiring is very simple because I'm just saying an instance of the scribe type class for app is just all of these abstract methods wired up to those implementations that I made in that file and everything sort of flows from that so I have defined the conquer the actual implementations in that files that when I go to wire all of these effects up together in this app type module it's really just saying take the interface and then make the method aligned with the implementation that I wrote that's just a function and so you kind of get I think a very neat way where in one module you see the way that your effect phul wiring in an application comes together without having to put all of that all of the definitions right in line here potentially clutter it up and so if I want to examine it I can maybe go jump to definition here and then see it but I don't have all of that stuff in there this is just basically saying this corresponds to that so when it's actually being run the process of running it is now that I'm in some kind of i/o context the main entry point for my application I make an HTTP manager I wrap it in HTTP config record I make all of these things for kata that my logger needs access to to work I put that in that configuration record I construct my whole application configuration record and then I run the app with that configuration record and in this case running the app is just running this effect full up top and so in in a real application this entry point would maybe be start your server start your event loop go do a bunch of stuff not necessarily just you know perform a little function but that's kind of more or less how this whole architecture is meant to kind of come together so before I go into what I think makes this interesting and potentially compelling and potentially a minor abuse of the type class system and Haskell which is the test functions are there any sort of questions this is kind of a lot of stuff - yes no and that's that's actually kind of nice the type classes don't care about IO and in fact they are not permissive of arbitrary iOS so when I'm pulling these in to be used in a function they don't imply that I can do our betray IO all they apply is sequentiality which is what monad gives you so none of these if I use logger anywhere allows me access to IO and it's why in the run module example earlier when I uncomment it out that lift IO line to do an arbitrary IO call it didn't work so this is basically saying this is sort of a sanitized interface within which I can perform effects that I will then wire up afterwards does that clear it up yeah cool any other questions all right so this is only I think useful when you this is useful because you want to maybe restrict some effects that people can perform but this is also useful because it gives you a nice little interface that you can sort of set up for mocking things and when you're testing and this is where I'm a little less sure about how good of an idea this actually is because I don't really know how I feel about mocking and so that's kinda want to get some feedback from from everyone here because I'm less convinced that mocking is actually a good idea the more I'd write code and so in this instance I'm defining a new wiring called a test app which looks very similar to the app type from before with some minor differences and so what I'm saying is the context that I want to execute my tests in is going to be different from my production context specifically in my tests I am NOT going to admit arbitrary IO in the tests because I don't I want to maybe constrain what I'm doing but I maybe want to have different testing backends for my different effects and this is where it's gonna kind of get a little weird what I'm basically saying here is I want to define different types of tests I want to perform for the scribe maybe for some tests I don't want to actually have the scribe do anything and maybe for other tests I want the scribe to actually take all of the logs that I'm going to be performing append them into a list and then get like that data structure back out afterwards to examine that the logging is done like done what I want it to and that everything sort of works the way I expect it to and so what I can do here is I can define a some type that just says the test scribe type could be either one of test no scribe or test scribe list and then when I have my test app here I can say that one of the type parameters being supplied to test app is of the type test scribe and so this now becomes a way that I can tell the Haskell compiler what instance to resolve it - based on what I pass to it and I can say get a type class instance that will either do no logging or gonna type lessons that will log to a list and when I'm actually putting it together here that's if you kind of squint you can sort of see what it's doing I'm saying that an instance of the type class that needs to log to a list is going to have access to some config that has the list scribes that has the list subscribes data structure in it and then I'm saying that this only makes sense to resolve in a test app that has the type of the flag telling me what instance to select called test scribe list and then here I wire up the instances to actually perform appending to a list in the context of that test and then down below I see the alternative implementation where when I've got tests no scribe I just wire up this implementation which in this case does pure unit and then blows away all of the actual test data or though the scribe data and four down here same thing sort of with scribe although the constraints start to sort of get a little hairy because now I don't just need the list now I need access to a test app that's got the test scribe blogger in there and then that's how the logirl gets wired up and this thing sort of gets a little weird when you're trying to have tests that can use certain dependent effects and how they get wired up but this actually all works I'm just not entirely convinced that it's the the best way to do it that's not overly complex so when it comes to actually running a test and I guess I'll kind of go through what some of that list scribe stuff is it's similar to the constraint that I had earlier that guarantees that I have access to some type in my reader context but in this case my type constraint is an IRF so a reference to modify some structure that contains a record where each of the elements in the record represents the things that are effect is gonna modify and then the implementation for this just becomes so like listing info basically says get act like view the state of the scribe from the configuration record and then modify the field info list by appending the message you want to log to the list of the messages already in the log state the rest of these kind of flow from that this is just using a little bit of lens syntax to make it easier and to make me kind of feel cooler about using lenses and the no logging implementation is just pure unit like I was saying earlier so when this is actually used when I perform a logging test and I can sort of show what that looks like when I perform a test of the logger what I'm doing is I'm setting up the scribe State by creating a new i/o ref to an empty state of my sort of scribe logger and then wrapping that in a configuration record and then passing that into my test app and this is kind of where the magics happening I'm using gh C's type application syntax to say when I run the test app fill in that logging type parameter with test scribe logger to tell it to log to the describe and then use test scribe list until the scribe to actually append a list data structure then pass it in the config and then run the log test function defined down below which just performs these logging actions and so this is all the orchestration that says given this state and given the type directions for how to actually choose the instances to wire it for your effects perform the actual effect and when we run the tests as it's sort of being done there it actually it works so when I compare the output state from my log to this test fixture that I'm defining they're they're equal and it shows that the effects were overridden to append all the logging statements in do some kind of state that I can examine and make sure everything works so that's sort of the value proposition of this approach I can use type class instances you sort of act as a mechanism for almost extensible effects and I can combine them in such a way that when I want to override pieces of them the actual test implementation it's not so bad the part where I'm not convinced is this is a lot of work and and specifically here when I'm going to put the test app up like you can sort of see how this is gonna maybe not be super tenable if you get a bunch of effects look I've got thirty effects here I've got thirty type parameters I'm passing around thirty type applications to wire things up that's gonna suck no one's gonna like doing that and then what does this actually get me and I don't have a good answer for that I don't really know if this is a good solution at all but this does kind of work and it does let me override things in this way this is mostly informed from Chris Penner's blog post mocking effects with phantom data types and another blog post that Matt Parsons transcribed from Will Jones talking about transforming transformers where instead of describing explicit mouna transformers he sort of creates these interfaces and these concrete functions and then implements them in instances on the reader t-that wants to perform everything so any questions on this and in addition to that any sort of thoughts or advice on this pattern in a way that can make it better or make a little more sense yeah yeah the question was are there any issues associated with generic lens whether it's compilation times or anything like that I have not had any compilation time issues for generic lens I can't say for sure because compilation issues with generics it's kind of gh C's deal so I assumed that there would be but and this is really neat as far as actual implementations of the functions for generic lens the guy that wrote generic lens is really smart and actually has in his test suite a plug-in for GHC that compares the generic the generically derived optics with handwritten optics that he has in the file to make sure that the compiler output the core output for GHC is identical to the functions that he wrote by an and so your generic lens functions if you use them should always be as fast and even beyond as fast it should always be identical and implementation to a handwritten optic so I really like that I think it's super slick and I think the ability to do things like where is it things like typed and whatever to get access to the yeah like I don't know I think things like this is pretty sweet because what's actually going on here is these test functions are polymorphic over the reader environment and the test app is actually everywhere polymorphic over the reader environment except for the constraints and so at no point do any of these test functions care about the environment its operating in except for the constraints saying that they only have access to the things that they need and so at no point have I constrained any of the things I need to run my test suite by a specific configuration record all of the individual test functions just care about can I have access to my particular thing that I needed to work any other questions comments advice but I'd say that this is very modern in Scala that's why I had to rust and we can see there for us to tack this final address I've got really tight glasses so it's not like we are defining our faces right so actually kind of interesting because you mentioned or four instances there are no orphan instances here and this is one of the things that I wanted to try to but I had a thing that used orphan instances for maybe a slightly nicer module structure and like three or four people told me don't ever do that that's a terrible idea that's really stupid just don't do that ever and so I got rid of all those because they were right and using orphan instances for module like niceness is not a thing that anyone should ever do if you ever think you need to use orphan instances and you have access to all the code you control think really really hard about that because you're almost always gonna be wrong orphan instances are sometimes necessary when you don't have access to the type that somebody is like providing upstream from you but like almost always they're not going to be the thing you should reach for first second or probably even third but yeah the the tagless final stuff is kind of what I was going with here I actually kind of started thinking about this because the code base I was working in at the time is written in terms of extensive extensible effects which is like an initial encoding and I was trying to sort of figure out a way if there was a good like an easy way to translate the initial coating of extensible effects to the final encoding of like final tagless or Witten Haskell's called the MTL style and this is sort of where I ended up I think it's maybe a little more complex than it needs to be but the thing I was trying to avoid was without doing this weird type parameterization stuff you basically end up with n squared permutations of your test app types for the N different effects you want to implement if you need to implement you know effect one with effect 2 and then effect 1 with effect 2 with effect 3 for all the kind of effects you have for the different mocks out implementations this is kind of I wanted to try and avoid that all right cool well I don't have anything beyond that so I think that's it you