Playing with Shapeless

scale.bythebay.io: Thomas Kim & Charles Ruhland, Playing with Shapeless

Recording: scale.bythebay.io: Thomas Kim & Charles Ruhland, Playing with Shapeless

you I was ruined and we're engineers ad come to call iterable we're at mark growth marketing automation company for big consumer companies and today we're going to talk about play formless which is a small library we've written that provides integration between Play Framework and shapeless so how many of you actually our users of Play Framework of curiosity okay like most of you it's interesting and how many of you our users are shapeless a good number of you right so whether you use play form work or not really the overarching goal of our talk is to show applications of shape lists for wide variety of problems having to do with data modeling and type level transformations which are pretty common programming problems and so we'll do a quick review of play for mappings and we know that much times we're going to move pretty quickly we'll talk about the library and how it makes working with for mappings easier we'll talk about how the library works using shapeless and we'll go through some of these features so quickly we'll review how for mappings work and play framework so for mappings basically allow you to handle web forms which is say to take a set of HTTP request parameters and bring it into a value and Scotland play case class and so when you have you might have a form with say three fields on it and you want to bring into this case class it also has three fields it's in this case we have you know string enumerate optional numeric and a boolean and to build the mapping you specify the field names which is to say the HTTP request parameter names as strings and you specify how that fields should be handled which is say how the HTTP request body should be brought into the typed field along with some validations and eventually into the case class so in this case we have the three fields we've been on the three string names and play out-of-the-box provides mappings to require that string be non-empty to require something properly parses as a number and to handle boolean's and so forth that gives us a tuple and then we convert the tuple into a case class and then we put that into what's called a form we're just going to hold both the mappings and the values after validation and then we can additionally use that same form data to render the HTML so when we render the HTML we want to have a similar set of metadata particularly around the HTTP request parameter names and use that information to create a HTML input field the same with the correct HTP request parameter names so a lot of your product from earlier with this and you've been doing this for a while and there's nothing wrong with that convention but we can do a little bit better so the three problems that we want to tackle or one is safety so as you might have noticed we brought over a lot that convention relies on using the same string parameter names and both your views and your controller it's a little bit brittle because it goes into a tuple and that tuple actually has to line up properly with the case class if I order the fields in the case class that code actually won't compile and thirdly which may be the worst problem it's actually just a lot of boilerplate so typically you don't care that much about the HTTP request parameter names and you you pretty much just want them to be the same as the case class names and in many cases the proper mapping that you want to use is immediately drivable from the type of the field so if we can do better than that and so with our library that the met gen eration the mappings and the form can be reduced to basically one line of code so we have the same case class and then we have one line of code that says what case class do you want to bring the form data into and how do you want to derive the field mapping so in this case we just take the fields the type of the fields and we immediately give you the mappings let's have defaults and then similarly in the view when you dereference when you access that field data you can do it by this bull and the symbol is actually type checked such that if you type the wrong name for the field you'll actually get a compiler error so next Charles gonna talk about how that works thanks Tom and so right so we have kind of this magical we have the kind of magical configuration here where we can just take a case class and somehow populate mappings for play forms with it and how do we actually go from the case class definition to to something with these where we can just access these fields by these symbols we're gonna kind of go into two steps one thing we need is we need given sort of a generic type to representation of the structure of the case class we need to be able to need to be able to process that into a mapping that play form can use then we also need to derive that generic representation with the mappings for each of the fields so I'm gonna cover the first part that I mentioned in time was going to cover the second so to get started we need to start with the fundamental data structure in a shapeless which is H lists so for those of you haven't seen it H lists are kind of like regular sky lists like here's the here's a value of one for example but if you notice we have different types in each of the elements of the list a normal list this would probably be typed is list of any but in shapeless with H lists each of the elements has its own type and so the type of the whole thing given an orange here is actually itself kind of a list so this is all the type level and this allows us to keep track of the type of each of the fields of the list as we use it in our program another thing we're going to need our singleton types of literals so what this is is here's a literal value of 42 that's the that's a value level at the type level shapeless allows us to sort of have a unique type representing that value and we can do this with any type of literal strings boolean's and also symbols which a little more obscure but they're kind of they're kind of like a special kind of string and the symbol case is actually to be especially important for what comes next so we're gonna use symbols singles and types of symbols along with H lists to construct a generic representation of the structure of our case class so here's just a field example so for a single field of the case class we can use a symbol to represent the name of the field and then we have this fancy arrow to the value of the field in this case my account and the type of this expression is field type of the singleton type for the key and the the type of the value of the field value now I'm going to use some short hands here to make it a little easier to read so essentially we can put multiple of these together just to describe the structure of an arbitrary case class we can say for example you know symbol X goes to 42 symbol Y goes to true H no and the type of that retains all the type information of the individual fields along with the field names so this is this is a completely generic this is completely generic and this allows us to have type information in our safe form which we can then use to safely dereference the field values okay so time like I already went over this so I'm gonna go through this pretty quickly so play provides you a way of composing these simple mappings into larger mappings defined for a case class so you know here we have a non empty text of mapping string we have an optional number of mapping an option event and we can even combine these together with field names of strings to produce a mapping of a tuple now we want to do the analogous thing with H lists so again we have our fields we can string them together using H cons and H nil and this produces a value of the following form so we have the field names which are associated with the mappings reach of the fields and so instead of the field values now we have the mappings for those values but where we want to get to what the play form actually takes as its in as its input is a mapping over the entire record so we need to do a transformation of the type from the type on the left to the type on the right somehow we need to move the mappings from the inside of those field field records to the outer level so here's where we have to do some transformations first we have to introduce a type class called make mapping sorry one second here there we go make mapping and what this is you can really think of this is just a little bit fancier representation of a function from some type T which is gonna be our field type to a mapping of some other type out which is the gonna be some arbitrary type associated with that field and we have a few transformations that we can do here so first transformation we want to remember those those key values associated with mappings of V we want to take that and transform it into a mapping over the the entire field the key and value pair so here's an example with name and mapping string next we got to two other transformations for handling H lists so we can take so HTML is pretty trivial the H cons case is more interesting we can take given some arbitrary H list H cons T if we know that H can be produce a mapping of age some type H out and T the tail can produce a mapping of some type T out then we can take H cons T and produce a mapping of H out cons to you out and here's another example of the following with the name field and the paid field from our account example so now why like all these transforms like why can we regular functions for this it's because at the type level normal functions won't keep track of the details of these H lists we have to use implicit definitions so I don't want to go into too much detail here but here's some actual code that does this so this is for the first transform rule and if you notice the apply function we take a field type of k2 mapping of T and transform it into a mapping of over the entire field type with T and similarly for the H con skate we have h cons t gets transformed to a mapping over h out cons t out and these two input parameters here are the assumptions that we made that h can be mapped to a mapping and T can be mapped to a similar mapping all right so putting it all together what does this give us we have a record here describing the structure of a case class example from before name goes to non empty text numb users goes to an option number paid goes to boolean now we can use our safe form class specifying the accounts case class and providing that mappings record and then this definition integrates the two so all right we have four implicit parameters here the first one takes our case class type T which is account and associates it with some L L is going to be the record type of the case last a generic shapeless representation as an H list with the field names then our make mapping line takes these input mappings m from up here and produces the output mapping type the mapping over the entire ageless record finally these two aligned parameters just to say to tell shapeless that these case class structure and the mapping structure are the same they have the same field names and so they can be associated with each other so knowing that we can basically go from this label generic owl to the map the output mapping and create a safe form of those output mappings guaranteeing that the field values are all there that we expect okay so Charles described how we handle mappings are provided explicitly by the developer but in many cases the mappings can be derived immediately from the types of the fields in the case class and so those are provided or those are created automatically by what we call defaults and so this is the syntax that we want to get to so basicly can just buy some of your case class and then you can specify a bundle of defaults which is passed through as an object and so this is how that works so the default themselves are specified as what shapeless calls a polymorphic function value which is a special kind of function that can basically switch off the type of the input and so these functions are provided as basically bundles of implicit that are named and bundled into an object and so in this case we have implicit that handle each type as a separate case and for each type say int boolean or optional we map to the appropriate mapping and then we can bundle those together into this object which has all the implicit and which handle all the different cases for all the different types including string and then we could take that object as a value and pass it into our with defaults method and the what default method accepts the defaults as this higher-order function and then as before we these implicit s-- label generic carries the case class into his record representation l then we have a helper type class called map values know which will take the higher-order function which accepts types like string and in and converts them rights mapping instances so you know nonempty text and so forth and called map values by way of analogy to you know ordinary maps and that map values function which takes a function applies it over the values and returned to the map against and map Bally's type class will map the function over the values of the record and give you a new record with keys but instead of k2v it gives DK to mapping B which is a result of applying the default function over the values now we effectively have an explicit set of mappings again and so we can proceed as before and produce apply our make mapping type class and a line to bring the explicit set of mappings into a mapping for the case class additionally you can supply both defaults and explicit mappings together and we basically just combine the two approaches into this very simple function we're just only one line of code if you exclude all the implicit s-- okay so the gist of this we won't go too much into it but basically you you take the explicit mappings are provided take the generic representation of the case class remove out the mappings that were provided apply the polymorphic function over the remained remaining fields then recombine the explicit mappings with the mappings that came from the defaults then recreate the mappings for the entire record then again as before bring it back into the case class boom then so there's a really cool project called refined which provides what are called refinement types for Scala and these refinement types allow you to express a type level additional constraints over the simple types so instead of just merely stringing in and boolean you can have well clinic example but you can have a string and require that the string be non-empty you can have it in and require that it be positive you can have a string and required to match it regex and so forth and you can express these constraints a type level and so play framework has an out-of-the-box sort of DSL at value level to express validations and constraints additional validation constraints that apply over your form fields so like you know a feel should be a phone number or whatnot and but those are not expressing the types but with refine we could express a similar set of constraints a type level and then we can use type of information to generate appropriate mappings that also have the same constraints applied at value level and we can generate those mappings automatically just based on the types so we won't go too much into how that works but basically it uses the same default machinery but we we additionally provide defaults for all the refined types as well as all the symbol types and so we simply provide additional implicit instances for all the simple types with the district whether what their refinements so we're going to how that works basically we generate we have another type class that generates play frame or constraint instances based on the type level constraint tags we won't go into this you don't need to specify records you can also specify named arguments through some additional shapeless magic all right so where does that put us now or what what have we just seen we have the ability given a case class to to specify a form mapping without specifying any other additional information essentially all you need now is a case class and provided these defaults and this ageless machinery we can we can generate a form for you that is type safe whenever you want to access the values of those fields and make sure the mappings always correspond to the fields exactly as they're specified in the case class so this is pretty powerful all the code is available to look at and our github repository here so feel free to check it out again going over some of the benefits this is fully typed safe it's less brittle than before like you don't have to worry so much about lining up those case class fields and it pretty much eliminates boilerplate as long as user default mappings there's no need to associate the string field name with the mapping name anymore or the mapping value anymore but of course there are always downsides with shapeless hum so definitely the tooling support is a little bit lacking IntelliJ will oftentimes give it to you some red squigglies in the editor we're factoring to be a little tricky because we use symbols to access the fields so if you change the name of a case class field that symbol won't be updated along with it however they will fail to compile so you at least have that compiler error messages could be a little obscure we definitely ran into that while developing this but I think once the library is sort of once the basic types are put in place using it is fairly straightforward out these errors and then there's possible it's possible that you could have long compile times although we didn't test it on large H list so we're not quite sure there but if you're interested in this kind of stuff or anything else Scala related please come join us at interval we 100% scholarship we love correctness type safety data modeling and we're the fastest growing SAS startup out there and we have plenty of really interesting scaling challenges people like that so yeah please check us out we have a booth and yeah any any questions [Applause] yeah we have not but that sounds like it would be another pretty good application for this all right the DC on the fourth issue if you had a performance issue with the form validation at runtime I yeah there really it shouldn't be because we the the actual result that we produce is is just another like instance of plays form we have a very simple wrapper around plays form it's essentially just delegates immediately to the form type itself so this should be negligible overhead most of the discomfort complexity that you see here with the shape list is done at compile time so when I consume this form in the controller I can do the regular poll yeah so we provided the same API on safe form and just delegates to private pinyin form that's unsafe all right I think we're out of time thank you guys for attending join the conference [Applause]