SF Scala: Charles Ruhland and Thomas Kim, Shapeless

SF Scala: Charles Ruhland and Thomas Kim, Shapeless

Recording: SF Scala: Charles Ruhland and Thomas Kim, Shapeless

[Music] all right so yeah welcome everyone it's interval so yeah we've I've joined the company about six months ago and in that time you know really kind of fall in love with the company culture the people here and also the products I think it's really compelling products so many times a little bit we're we're building a platform a SAS application for growth marketers so you know be served marketers at companies like Spotify or country you know b2c companies they're looking to you know retain their users you know and get more users involved you know kind of keep engagement going and so they need a lot of help with that it's pretty involved got send emails gotta send text messages push notifications and meaning help them track and manage all that so you can schedule email campaigns you can set up user lists to send out you know campaigns at different times you can put users through workflows you know where like an onboarding workflow for example when you join when you join the company for the first time you get a sense a series of emails to you know let you know about here's what's available in the products and all this you might imagine requires a lot on the backend because not only are we having to support our direct customers who are the marketers at these various companies but we need to support all of their customers so all of their users who get these emails get these messages they're gonna be opening the messages reading them clicking on the all these events go into our system and we track them and make them searchable by our our customers so they can get analytics on how their campaigns are going so definitely a place where scale is very important we use elasticsearch postgrads Redis and rabbit with you in the backend data services and are looking to scale up tremendously this year so we're going to be scaling the products and scaling the engineering team so match so we are definitely hiring if any of this sounds interesting to you please let me or Melanie a recruiter know and yeah we're looking at 2018 be like the most awesome here yet at the durable so thank you oh that's right yeah we should just go okay I'm Tom Kim I also work at an herbal and today we're gonna talk about tight classes so we gave a talk of scallop by the bay around shapeless and here we're gonna give it more some it sort of tutorial around just using which is more sort of basic level or more detailed level around using type classes in general so how many of you already using type classes and your code and how many of you would like to learn more about them okay perfect well this is for you so shapeless so type classes I like to describe them as the most important thing in programming since object-oriented programming which sounds hyperbolic maybe but actually no not at all so and the reason that's so important is for two reasons one is that type classes saw a large class of problems that are very common in programming overlaps quite a bit actually with the type problems are solved with object-oriented programming in terms of allowing you to program generically and with a level of abstraction over abstracting over structure and the other reason is that type classes in Scala are gateway to many of the more advanced Aqaba things that you probably heard about libraries like like cats it's co-ed and libraries like shapeless and so if you want understand those libraries Union type classes first and sort of have used them in practice so in this tutorial and there's very excellent resources online for this including the type restaurants guide to shape lists put out by underscore in this tutorial were will give you introduction type classes and how to use them and how they work alright and I apologize any man's I have a cough so if I hear her under sounds coming from me it's okay so type so actually you want to hold this Charles and I are very affection of each other work as a team not just because we're the mics have to work that way yeah okay so so we're gonna do a live coding demonstration here I like that way better but okay alright so so to demonstrate type classes we're gonna start with a very simple example we'll start with a very simple thing that one might want to do which is great to find an interface for printing things and so that's commonly called the show type class and if you were to do this in sort of an object-oriented way you might define an interface show and define a method on that called show that returns a type string and already when we do this we see we run into a few problems with the traditional approach using interfaces the media problem is that you cannot have Show be implemented by your primitive types boolean string and so forth because you don't have control over those types type classes allow you to do a lot of things that you can do with interfaces but they solve three three problems that you comple faced with interfaces the one problem that you see here is about modularity so because when you use interfaces or extension or inheritance then you necessarily have to redefine your class hierarchy and that won't work for primitive types and and there are a couple other management style classes to have but we'll see later on they work better respect to modularity they work for primitive types and they compose in more powerful ways and traditional interfaces and there's so in a sense type classes let you reuse the type system of the language that you're working in which you can't do with original interfaces so so let's see how we solve this problem using type classes so with the type class has three parts the first part is a type class itself and so to turn this interface into type class it's always gonna be a generic type class over type T and then there are the so called instances of the type class and so in this case we'll define a few instances actually I'll need to make it and solve it kind of case so you define type class instances for each type for which you want that type class to tie class support for that type so in our case let's say that we want type has support for some primitive types so in that case we define the instances for the perm types of one support so let's say when was support you know in de jure and string and boolean and we would define the instances by creating a basically a global instance of value of the type show for T or for that particular T that we care about so in this case in yeah and we make it implicit you'll see so just takes so in this case we I have you rated yes and we don't leave this like that right and so so the second part is the instance and the third part is the usage of the type class and so in this case we're going to define a method that uses the type class so this is a method that makes use of the type class so if you haven't seen this syntax before there's D sugars to the following so in Scala type classes are all done by way or all done by convention by way of implicit and so let's call this and then we can call that with for example that return so again there's three parts of the type class type quads itself the instances which are defined as global and place two globally available implicit and then the usage of the hat type class which requires the implicit as a parameter in particular the implicit to be available in the usual way according to implicit scope and if you want to find further instances then know here oh just copy this so we can also define instances for for other cases and then we can use those instances now we can say that true no importantly that if you pass in a value of a type that's not supported by the type class meaning no implicit instance exists for that type class and your code will not compile so in this case if we pass in so let's remove the instance for a string for example we pass in and this coach should fail to polish or how do I look in the worksheet doesn't do anything but suffice to say that code doesn't pile they'd be good to see the error message [Music] it looks like it's showing an error in the upper right corner right here yeah no information well just so you know this doesn't compile so so far why shenyu seems like kind of interesting but maybe not that powerful so we'll start seeing some more advanced uses of it so one thing the type classes that can do that's really powerful is that they compose and so a really common thing that you want to do is you want to find an instance of show for option what's interesting about option is the option is parametric so it takes a type parameter so how do we do that so to define it instance for option you need to do the following so basically you want an instance of show of that type but needs to be generic so we can do is instead of using implicit values implicit def you're plus a def can take a type parameter T and then we could provide implementation of that type parameter but in order to provide an implementation of that type of show for option of T we did in the instance of show for T itself and so you can express that requirement by adding an implicit argument to the implicit dab called T show and now this implicit def will be available whenever T show is available in particular whenever shows available for them the primitive types or it can also work recursively so in that case since we have a helper method here we just do this and we'll just delegate to do strength so now we can do show of sum of five and this doesn't screw it out anyone know how to show errors so much something must be failure after that yeah it was oh my gosh I knew new worksheet know that there we go okay I think we're back in business so this worked up to here so didn't like the option oh it doesn't like that you think sure I said only the options and what's missing that option here show option show option of tea yeah you gotta use some yeah oh the tea show it's a t-shirt show shall I turn this one yeah welcome back to that this should work okay so so with that example work that you see that type houses compose and so the way that them plus the search works it case it's not clear when you write code that uses his head class it looks at the required type of implicit that you need so in this case we're required when we call this show function that there would be a show instance for that T sounds going to search for an implicit of type show of T and so we have instances for show char and show boolean we have instances for a show of option of T where this implicit is required and so to satisfy the implicit requirement it not only searches for the implicit says that define us vowels but it calls a search for implicit instances are defined as deaths and recursively search for the plus it instances are required to satisfy that def that'll be an important piece of functionality will see later so now we're going to take in a sort of an aside and talk about something that seems somewhat unrelated which is type level functions and so type level functions are pretty commonly used you've probably all used the type of function and so an example of a type ol function would be something would be any type constructor so if you've ever used list you've used type level function list is a type of function that takes the type t2 the type list of T and so sort of self named there's a couple other sort of turbulent coatings of temp little functions within Scala one is type aliases and so you can have a type alias called say foo of T and that will give you a new type which is the pair of T with T for example and this is another somewhat trivial example of a type old function you have to find a type of function called foo which takes as a parameter T and gives you a result type which is the tuple T of T now there's a more sophisticated coding of type functions in Scala and we call these dependent types and they're used throughout libraries like shaped lists and they they fit pretty closely with type classes and so we'll show how that's encoded so an example of an so a tight little function is it is a type constructor that takes a type as a parameter and then yields a new type so there's another way that you can Co type all functions in Scala and that's using path dependent types so you can have a trait and for example we are going to define we're gonna call this make option for reasons that we'll see it's going to take as an argument type parameter called T and it's going to yield an output type that depends on T and that output type will be contained and the type member called out so what's going to happen is that we'll have instances of make option which we should declare for particular values of T or for particular types T a particular output type out and this output type is defined as a member as a type member of this trait now pretty commonly when you have or almost always when you have a type of a function you have a value level function that goes along with that level function and so in this case we'll have we'll call them at that apply it's going to take a value of type T and the return of value type out and so as an exercise as an example the type of a function that we're going to define is going to be a function that says for all types T give me option of T unless it's already an option so you may never have done that before and we're going to show you how to do it and so you can do this so the output will always be it's just if it's already wrapped and it won't do any more wrapping this basically idea so this is again is done by way of declaring make option as a type class and so in our type class we start by defining an instance for things that are already options so if it's already an option then your appetite is simply going to be the same as what as the input type so since is for option option as a pram is parameterised we need to make that nipples a def so we'll just call this option it's option so basically when you're searching for an implicit instance of make option for something that's already an option then use this instance of make option whose output type is going to be just that same option again and the value level just make it that identity so now we have a type class which the the implicit instance or the instance for that of that type class for four types that are options is simply to return option type next for all other types we want to return that type wrapped in an option so we use one trick here which we'll see in a lot of libraries will move the implicit instance for the fallback case into a super trait by moving into super trait we basically tell the compiler to use this in the fallback case meaning use this instance of higher priority than the instance that's in the super trait and so in the super trait we can say a non option type should be wrapped now this is necessary because this case is just going to apply it on for all types T and so you don't want to conclude with the definition that's specific to options so now if we do for example so now we have this type class we have instances so if we try to use it so let's say it's not actually have a function call make option which takes a type T and then is going to give us a value that's always wrapped in an option if it's not already wrapped or if it's already wrapped as an option than to simply yields T so to use this we require that a type class instance exists for tea and the permission is always going to exist and importantly this function requires make option as I plus a parameter as a and then it's going to return again this is a dependent function so the type that it returns depends on the input type so to express that you can say that the return type is the type member out off of this implicit parameter so clearly this function is dependent so it works yeah we can make a summer option the other way so if we call this so the claim is that we call this function with a simple type it will return that type wrapped or that that value wrapped in the in an option change the mode traditional mm-hmm it hasn't been going line by line at this point yeah there's something wrong here just try reloading it I'm not commenting these hours oh okay cannot find a Plaza for Mac optional okay this is real code and sometimes it does like a pile I don't see let's try to get here make option events we did this oh we know what it is I think this is conflicting with the other worksheet maybe close the work she did reopen it I think that could be the share some cash the state in there thinks that oh I see okay so let's try weird if this worksheet is better actually oh actually no here's the problem because it's a worksheet you can't define the treat and companion object together see see it's giving you a warning yeah changing yeah I'm taking the important yeah turn it off oh yeah yeah yeah here we go ah there we go alright so you can see it wrapped it in an option and if we did this properly then if it's already wrapped in an option then it'll actually just return the value I've wrapped it twice you've changed back to option Oh asked me actually just back yeah okay the only little priority is coming yes yeah down at the bottom t came the change adapt could be it yeah that could be it yeah sure this work will be trying to locally but ah yeah good catch okay so some gotchas but you can see their overall a goal so we get we have this type dependent function given by type class which varies as a result type depending on the input type and so if things are already an option then it doesn't wrap it again yep yes there are there are definitely ways to make it work with some offhand I don't think I can fix it live here yeah there's a couple different ways to do a pretty sure the direct way to do it would be to with variants so I'm not going to experiment with it here but it can be as simple as just adding a plus here no I don't think that will work exactly I mean it did okay well with variances things do start to get hairy with using tech classes I'm pretty sure reloaded so that seems like one small step but combining these techniques you're gonna work we're gonna we got work ourselves up to doing all the powerful things you can do with shapeless and other type class libraries and so so just get into shapeless and you'll see why these approaches are so important for a library like shapeless we're going to take a small interlude to talk about about H lists which are really important shapeless and so so yeah the idea behind H lists they're kind of like tuples right where every element of the tuple could be a different type Y shapeless has H list though is submit there's a sort of a basic common way to refer to all of the values it's like it's as if all tuples had a common base type and that will come in handy as we'll see with some of these implicit type of class definitions so we'll actually just take a quick peek at the shapeless source and just a quick peek at the definition of H list so H list is a you know it's it's to find almost very similarly to way list is to find a scholar as a linked list but it varies on one important detail which is at the cons case you have a head and a tail so actually if you look at lists in Scala [Music] you know it's an abstract class and it's got two cases it's got nil and cons and so in the cons case there's one type parameter and you have the head is up that type and the tail is a list of that type in the Ageless case you have to type parameters you have a type for the head and you know type for the tail and the head is its own type and the tail is is you know recursively defined as another ageless type and so in that way age list is similar to linked lists except it preserves the type of all of its elements and so when we use an H list so like this H list here it'll have the type just wish the redbull problem yeah it'll have the type of you know string boolean and H nil cons together so it's the never actually worked actually because that yeah yeah that's interesting okay so anyway with this H list you'll see it's preserves the types of type string boolean and the HTML and so if we look at shapeless shapeless provides an important piece of machinery called generic and generic and by way of a macro provides a transformation from any case class into an ageless representation in that case class and so for example we have case class account which has two fields classes so it keeps clearing the screen each time and there's you can actually invoke generic isn't apply method on it which returns the application of this of the generic type class to account and you can see it has so if we look at the definition of generic you'll see it's [Music] it's a type class and so it takes a type parameter which we've all seen before now with type classes and like other types I've or like other dependent type classes it has an output type that call that's called that's called wrapper so this is another example of a dependent type sight class and that value level it provides functions which transform instances values of type T to reprimand from R upper back to T so repper is the ageless representation of the type of T it's the generic format of it so if we go back to our generic account it's gonna produce a type of generic where the word them the type member repper has the desired ageless types in this case string cons boolean cons nel and does that by way of macros so now one thing that we want do the show type class that we weren't able to do before is create an instance of that type class for all type classes I'm sorry for all for all case classes and so you can see why that would be useful we created instances for int and string and boolean we created instances for option but what we really want then the day is an instance of show for all case classes and shapeless let's you do this and so to do this I might show them just copy these in we've already implemented there's certain pattern for how this is done and what we do is we create implicit instances for two cases H columns and H nil so then plus an instance of show for HTML as trivial and returns empty string then plus the instance of H con of a show for H cons requires implicit instances of show for the head and tail and then provides an implicit instance of show for H cons T which implements show delegating to those other show instances for the head and tail so this is exactly what we did before when we had when we were composing show instances for option it's just like somewhat more sophisticated so again it's also recursive and these are called inductive implicit and yeah yeah like one way you can think about this you know like with sort of ordinary style programming where you might do a pattern match list I have two cases for the cons and the mill cases and the pattern match this pattern here is is the same thing it's a pattern match but because we don't have you know in a normal pattern match you don't really have you can't have different types on the different cases in order this is for the different types in the H list the different cases have to be different definitions with different parameters like generic parameters so you've taken the ordinary pattern match construct and sort of lifted it to this implicit search level where the Scala compiler will actually end up sort of traversing over the structure of the H list as it does implicit search yeah and so the reasons called inductive implicit sir because we're building up the implicit instances one by one starting from a chanel then going from a you know a single element h list to ln h list and so forth and the two element h list instance that the case for the tool mah list delegates to the case for the one element h list and so forth should we try it out yes I'm sure it will work perfectly so actually there's one more step that we need to do which is we need to convert an arbitrary case class to generic representation and then and then apply the show for the Ageless representation of the case class and so we just define a helper method actually I'll just copy this and so so for case classes we can implement a show method that show method requires that there be a generic representation of that case class and then a show instance for the genic representation we haven't talked about this ox thing but as we mentioned these these type classes which would depend function types they have an input type and an output type and this ox thing basically lets us extract the output type which we're calling our in this case from the case class so basically that thing that's buried is a type member wrapper we can we can extract it as R and then we use conveniently in the value level functions that provide about the tape type classes are always exactly what you need to implement the instance and so in this case we need to convert so given an instance of it of the case class T we convert to its excuse me it's its generic representation R and then we take the generic representation and invoke show for that generic representation and so if the worksheet cooperates with us then we should be able to do well we're gonna buck em so you can this could be either implicit or explicit it's try and check is it in repla mode reloading think this works better when we put these inside the you know I think he's having problems finding you but these class ends this is out of the worksheet they works better if I put these in the like that I don't think Malcolm not only not working this works you know there might be surprised that worked so unfortunately well to test this by the next time but could you just manually pass them to the parameters is complaining about right now but they should cons can I you should know okay sir well with it needs to be case class show because we have to pass a class into it that's the oh we're missing the parameter list for the case class you have to have a parameter of type T up for [Music] oh it's 8th class show - oh that's what it is Kay's class show - oh right right you have to do a bracket account that's right yes that's right it's just show it's just show yeah I mean I want to make it explicit yes we just have to we have to just user we already have a method called show that gets the type classes yeah I if I made this implicit I could do that for this I didn't want to your try that though yeah it takes it takes a parameter out of there use the show method directly instead of case class show you this this is defining doing implicit def excellent like this implicitly elicits death yeah just use show yeah as long as that's in scope is that did we do we come to that out up further up though yeah back out from before of course now I might not have a man thing is done using worksheets for a lot coding I would stretch mine implicit values for yeah okay well we can move on to because there's more just like odd yeah yeah okay well suffice to say that this will give you a show instance for show and it'll print out the case class using those instances so now to talk a little bit more about shapeless okay so the last thing we want to show you or that so H lists have many of the usual methods defined on them that you would expect for say a seek or list you can do ahead and last a cool thing that you can do is you can do top one but things like drop it knows statically the the length of this age list and so drop one and two will compile up a drop three won't and what we want to show you was how these things are actually implemented by shapeless there's also a cool function on ageless called select which will select out of an H list the field of the type of of a specific type so if you call select bracket string then I'll select for you the field of type string which in this case is name and so all those operations on H lists are defined as type classes interestingly and so if we look for example at the function select okay let me do it right here so you feel like a deaf select you'll see just like the code we've been writing so far it delegates to type class and then simply invokes the apply method on that type class if you look at the definition of this type class you'll see it's implemented just like the code that we've been talking about it has cases for the H cons case and for the like isolation ill excel in ever compiled never work and this particular type class has two type parameters a type for the H list and a type called you for the type that you want to search for in the h list and the instant as it provides an instance for when and then it has a prawns case for head and tail for where the head and you are equal that's defined right here and in that case it simply returns you the head and it also has cases for this is essentially get defined as a fallback where head and you are not equal and in those cases it recursive over the tail and just as before these instances are only available if the h list contains some case where u is a member of the h list and otherwise it will fail too it'll fail to find this implicit and all fail to compile and so much of the code of shapeless is simply implementations of these type class instances is that one of our main goals and storia live you enough knowledge about how type classes work to understand how to read the code behind shapeless and there's many many more operations that provides things like a lines so that you can take two clay case classes with the same names in different orders or fields with the same name but in different order and map them to each other ways of lifting case classes into into other type constructors or type classes and more broadly you can use these approaches to implement all kinds of generic programming solutions for all kinds of generic programming problems so these are core commonly what things like configuration when you deal with a STS exactly and yeah so that concludes our tutorial any questions great aside from the worksheet problems I hope you're excited about using type classes in the future [Applause]