SF Scala Meetup: Aleksander Rainko: ducktape - holding Scala's productivity together
Recording: SF Scala Meetup: Aleksander Rainko: ducktape - holding Scala's productivity together
foreign ER I'm the actual duct tape I also work in scholar C I've been talking about this thing here and what can it do so someone joined the first um so first of all I will go on a quick round about Json apis and how cool they are to to and not boring there are to produce and and then I will introduce you to duct tape what can you do what's what what it really is um I wouldn't go on to talk about ideas and code jump like engineering Scala and then we'll put all the above together for a nice cool example so let's start with the story the story is that moving Json from socket a to Second B pays the bills like 60 of the time at least for me and let's read the hash show for software development because there's a lot of like cool work to do but there's also not so cool work to do like a Json apis right and but we all have to do it on some point in their lives uh yeah and then also really really mundane to create and do um so I came up with a list of things that we need to do before before we like move on with our lives uh when I when we're creating a rest API so first of all we create the root definition uh with a DSL over Choice uh so that's your archive https and this HTTP 4Ss uh what else is that I don't remember uh so yeah there's always some code to write there right and it's always not like really much fun to write at least for me but yeah so next up is the request response model so like all of your probably all of you all of your endpoints have a requests a request model and a response one right something comes in something comes out uh so yeah so that usually takes form of case classes in Scala you also need the additional not for your body but some things to do with actually writing writing up uh the serialization sterilization Logic for that so that's another thing but then uh usually our apps have like a domain model and uh and the API model which is created so we also need a kind of a bridge between those two words right and there's some more code right there and given that our domain models are not like usually are not like one-to-one mappings between each other and they also involve involve validation parsing sanitization Etc so there's even more work to do here uh but then once we've done all of that we should probably also document the things we just did right uh and then once we've actually documented we need to remember not to break it uh so yeah another thing to keep in mind and then we're free to apply the business source we actually can do what we came here to do uh so yeah you can see there's a lot of things going on here a lot of things that are like prone to error at some point uh so what if you actually didn't have to do all of this what if we only had to do those basically those two things so we just document the thing we want to do and then we do the business thing we basically move on with our lives after that so yeah uh that's why like I will try to show you today and the first like ingredient in this in this all of this is duct tape so what is tactic a duct tape is a micro based library for automatic and configurable Transformations uh what that means for like the user is that for example you have a case class that has two case classes that have basically the same Fields between either in like on the order is different maybe one of them has more for than the other uh what that means to you is that you can just write it online and then basically Maps between those two data models uh that some to some of you that might sound kind of like chimney because it is heavily heavily inspired by chimney uh and what this chimney chimney is also a sky library for bullet page fully free and type same data Transformations uh so yeah duct tape is really a slightly different take on things first presented in chimney uh and also that episode exclusive so like skeleton exclusive rather what that means for me is I get to use all the nice really really nice micro things in Skytree and don't have to really worry about Scala 2 uh so yeah I guess that's nice at least for me so what is it made of it's made of type classes mainly uh like one type class in particle it's called Transformer and the Transformer like the name implies transforms things right you guys you so any transformed transforms things in a total way that size there's no way of that of a transformation that came from a Transformer through basically fail at runtime where you can fail though is a compiler time but that's completely fine for us right so Transformer is the Primitive that everything else is built out of so there are instances for the most common Transformations available out of the box so for example things like turning a list of things into factories right so maybe wrapping something into an option just by just counting some it's all there just for you waiting for you to use and then there's also automatic derivation for case classes and minimums so like the meat and potatoes of the whole Library really uh yeah that's the example actually so uh the first one like just just imagine that it's in every single like uh example I show you so just from now on that won't show up so in the example we have uh set two sets of case classes they're basically the same only the like the as I said the uh field order is different so if you want to for example uh transform between attack one and attack two sorry attack and attack two uh with duct tape you just basically this you will use the two method and what is the two method it's an extension method that is brought in from duct tape and what what it basically does is it materializes a Transformer between those two types so between talk and talk two and then immediately applies it giving you the transformation right and the code that gets generated underneath looks like this so there's no like magic really magic things going on here it's just a primary constructing invocation of the on the Destination type with all of the fields basically uh written over from the source type and those also suffer from nested Transformations which are also not uh you know I wrapped in order to customers they're like unwrapped in at compile time basically so yeah so that's what the case class derivation does and the rules for that is basically uh that the when the source case class has all of the fields of the destination case class name ice and the types that correspond to those fields have transforming instances in scope then the the library is able to derive a transformer for you for free that's also support for enums like if you see enums made of singletones and I'm actively working on supporting like proper adts made of not only Singletons but also other products uh so yeah but for now each only works with announcement of Singletons so another example to enumz one of which has basically a subset of the cases of the other one right so we can just say this so we can once again use the two method and then code that comes out at the end is just this it's just an if else statement on the uh on the uh Children of the source enum and basically matched by name and the uh the instance from the other enum from the destination enum is returned as the right hand side of the if statement uh so yeah so that's what uh what we can do like uh straight away uh what about what about the cases where some things don't perfectly line up like they often do uh in the real world so once again we have a set of two case classes uh one of each other person the other one is a serious person and the only thing that the Finish differentiates them uh is the additional SSN field which stands for the social security number doesn't really matter so if we do uh if you want to transform a person into a serious person right now uh the compiler would yell at us it will just tell us that no food name there's no field name SSN found in person right so what can we do uh we can use into into uh is another thing it's basically a transformation builder of sorts uh so use into just like you would use two only with into right it's enough to uh so then once you have your transformation Builder you can you just start applying the transformation uh config transformation configuration options whatever to actually fill in the blanks right so in this case I fill in the blank of the SSN field so fill that const so the first argument here uh pig is the field from The Destination type that we want to fill in and the thing on the right is basically the constant value that will be filled in with so there's still that cost and there's field that computed which once again picks the field and then applies a function from The Source type to the destination to the field type of the destination field and there's three named which basically allows you to take a field from the other for the other case class and just put it in kind of like without applying other fractions basically uh weaker version of computed in this case okay so that's that's what we can do when like the constructions are private and available in scope what what what can we do when the actual deconstructor is is private right because as you remember like in that in that example before you know that the automatic derivation invokes the primary Constructor but the primary Constructor is not always available to us on the call side so what can you do no yeah there's another example so just basically in the same set of case classes uh the only change now is that serious person has a private Constructor and serious person also has a smart Constructor in its uh in its companion object so what we can do is to use via what is via VIA takes in a method reference let's say that actually will use that so that tells the library to use that method instead of the primary Constructor so in this case it will try it will try to fit in the arguments sorry the fields of per person into the arguments of the create method right so it fit in the first name it will filled in the last name well it will fail is the SSL field because once again person doesn't have the SSN field so we once again have the compiler again on us that no field is named SSN is found in person right so once again we can create a transformation Builder called called via the into via method at this time with other transformation options they are basically the same as the field ones there's const there's computed and there's renamed one thing was like pointing out is you actually get um type completions here and to do to how uh I believe in scale 2 there was a feature into the introduced that allows Ides to auto complete structural types so that's cool I guess uh so yeah so that's what you can do when the Constructor is private what what about even more elaborate cases uh when you're like case classes are made of maybe other case classes that have Smart Constructors or maybe are made of new types that basically don't expose public Constructors but only expose smart Constructors that return options either Etc so uh once again that's also possible we just need to do some additional work and the way it's possible is there's another thing called soluble Transformers which are just like Transformers only they can fail at runtime actually uh so yeah so our Transformer introduces a wrapper type of F and that wrapper type is the type of the basically the wrapper of the transformation so think of it as maybe returning an option on an either so F can be an option can be an either can be anything you actually want it to be um so yeah so the way uh soluble Transformations work they kind of work in two flavors and those flavors are called modes in this case so first of all what is a mode so a mode basically tells the automatic derivation mechanism which way of like composing your valuable transformations to use so there are two modes there's a accumulating mode and there's a very fast mode but also modes also um kind of serve as a a provider of operations for those like abstract abstract Cropper types so we need uh some as a certain set of operations to actually do the transformation with the like with the abstract type right so the the only abstract operations we need is basically pure which basically which wraps a ordinary value into that wrapper type so if f is option then pure will just go summon it right uh there's also the map method which allows us to uh work on the values inside the flapper type and then we move on to the flavors of of mode so one of which is as I said accumulating and accumulating introduces another method to us in the product method and the product method uh takes in two values both of which are already wrapped in a rubber type f which is supposed to mean that we can easily like treat those two values as independent Transformations and we can run them in codo Point parallel mode which will allow us to basically um accumulate the arrows those those things produce right so that's that's how we get accumulation semantics uh infallible like Transformations and there's also the fill fast mode which introduces the flatmap method uh so development method takes in a value of rapidly on F and a function that goes from a to some f of B what that means is that each soluble transformation is actually in a sorry in a fraction chain is actually dependent on the value that came before right uh so you cannot have all the transformation after a failure actually occur because they cannot exist because the call in flat map failed so that's when we get paid for semantics uh so yeah a quick example of that is once again we're back to personally serious person this time with a Twist uh the service person has a like a new type over the age and it's filled and its assassin types so the age new type it sounds really any type but you know the H new type exposes are soluble Transformer that does some validation on it so it basically checks if the value is bigger than zero if it's uh if it's not it gives you a nice error message if it's if it is then uh it just wraps that in the uh in the Constructor uh and there's the SSN uh thing new type that doesn't expose uh Hollywood Transformer but exposes are the field value of some kind and only exposes a smart Constructor so no fiber Transformer just yet so yeah so to actually transform between a person in a serious person in this case uh we also need to provide the Transformer mode and Transformer modes need to be present at color site not really in the colors in the implicit scope of the color side basically right to for the micro to know which uh which expansion to use so in this case I use the accumulative mode for eithers for Ides with uh collection on the left side so it can be a list it can be a vector can be whatever you want and then if I were to call a person into serious person I also call the follicle combinator on that which puts us in into another kind of transformative transformation Builder valuable transformation builder in this case which we can once again fill in with our transformation options so all of the aforementioned uh Total Transformation options are available out of the box so const computers renamed their they all work here and there are two variants on The Total Transformation options so there's foldable cost which actually lets you up Supply a valuable value on the right hand side so as you remember ssn.make returns an item of list of shrink or SSN so that's what what type is expected here and variable compute that allows you to supply a valuable like transformation from The Source type to the type of that field wrapped in the type that is actually in implicit scores wrap in the type that the mountain scope has so yeah we oops Yeah so a code like this expands into something like this this odd I've got on the top that we don't really care about uh the the thing we care about is right here so uh an accumulating transformation uh is actually a chain of product calls on each on each soluble transformation right so the micro would uh if you have like more Fields here it will be like bigger but of course the supplementation uh so uh you have a like a product call on the ssn.make method and on the Transformer of age that was that we put in the companion object of age so that was actually already provided to us by the implicit search mechanism and then that gets followed by a map call and that map call right here right uh the structures the Tuple that that actually gets created during the expansion uh and uh and at the end invokes the public Constructor of serious person with dividers from that Tuple right and the type that gets returned and the end is either list of string serious person uh and yeah and if you were to swap out the transformation modes for fail fast this time uh that would look like this so once again some code that we don't really care about at the top and then the actual transformation is just a chain of flatmap calls so we get actual like um very fast semantics here of course with the last map call at the end it's kind of like uh basically what uh for comprehension this should goes into so yeah with that in mind we can move on to ideas so idls they stand for idiot stands for interferations interface description language and uh example of that is open API which is a a yaml based IDL for describing Quest apis what I'm going to talk about like uh doesn't really concern only open API it concerns like ideas in general for example for us but yeah I use that as an example here uh so yeah what are what they are so they're usually decorative and focused on a single thing in the case of open API at least they have codes and support for every possible language on this song so basically if it has a compiler it probably has a code generator for open API for for that specific idea rather for open API in this case um open API has great scar support in the form of guardrail and it's your basically your source of Truth for both implementers and consumers so that's basically your whole documentation right here uh and yeah and what goes into an IDL and in this case open API so in this case uh we create endpoints by basically creating some path so in this case I created uh an endpoint called slash conferences and that creates a new conference that has a requested body of create conference we looked at in a second look at that in a second and it has some responses a well-defined responses so 201 for the cases then that it all works or 400 when there's some validation errors basically an invite in entity so what that will give us after the like cogent step oh no sorry uh also there's like the object definition so create conference is an object with three fields that doesn't really matter all of much here uh so yeah let's move on so what does this give us it gives us the API model right uh so in case of guardrail it will generate something like this uh so all of your like object definitions from open API become case classes uh and all of the case classes have encoded and decoders for Json in scope for free a code generated right yeah there's also one for date span and one from the validation errors uh it it also generates the unfound part I like to call it uh so the answer part is the part where you basically create the actual definition of the root your actual the civilization logic and serialization logic right so it all goes here and it's all generated for us according to the spec we of course given it at the on the start it also generates the response models and at the very at the very bottom is the actually useful part so the handle method it usually generates a Handler method for us to basically Implement and the handle method is the is like our wrong uh business logic so uh only thing we need to do is we need to implement that and we get access to the already the serialized body of the uh of the request so in this case create conference right uh so yeah so once that one once you have that in hand so uh now that we know like how the um how the code is generated we also need a way of actually uh structuring our domain to uh to be as like as automatically be transformable as possible let's say so yeah uh so how can they do that we can do that with new types right so scatter tree is nice enough that it exposes the opaque type mechanism which basically allows us to create a new type Library quote unquote in like 10 lines of code or even less if I didn't have to break my code like that uh so yeah in this in this example I actually use refined to materialize my uh validations in an integer in an easy way and then I also use source code for some nice uh method metadata that I use in my error messages so yeah so like what goes into a new type in scholar tree it is comprised about opaque tab inside it usually has a smart Constructor so in this case it has smart Constructor that materializes in the constraint um and then I also specify I also derive uh Hollywood transformer for for that new type and by just invoking the smart Constructor so nothing really fancy going on here so yeah each so like the whole point of that is that for the each for each new type we create to have a trans in portable Transformer in scope in implicit scope I mean uh and yeah all the user has to do is to provide a refine tab that redescribe the validation so let's get to modeling uh as you remember uh we're trying to model a conference like a conference insertion finger or something so I call First you know domain model would look like this so it's a case class of three Fields uh all of those fields are new types besides date fun I believe so yeah so name is a new type validated uh so they think I created on the last slide and that wraps a string and it has a validation a constraint called unsurprising string and an answer price exchange is just a trimmed and an empty and a Max size string of some sites right so in this case it's in an unsurprising string of Max s20 uh city is basically the same update spawn uh is not a new type it has a case it's just a case class with a private Constructor that also exposes a smart Constructor right so let's put all of the above together so for our endpoint definition like the one we saw before it will look like this so we create usually something called Roots I I mean I call it Roots right uh so we create a conference routes with a confidence repository that thing is basically outstanding for business logic uh in my case it's just a basically a call to the database to create the conference uh and then I extend Handler of Io Io is the effect type of my choice here yeah I provide the accumulate the sorry the Transformer mode I want to use so I want accumulating transformations to for my users to actually see the things they didn't do properly um yeah and then move on moving on to the actual implementation of the create conference method so once again we get access to the body right so the serialization body which is create conference uh and like the whole point of that is to transform the create confidence thingy into our domain conference thingy so we do body interconference that's volleyball because the transference will be valuable and then the only the only field that needs like a kind of help to like the only thing that the library needs to help in transforming is just dates fun so we can just use valuable computed to compute a date span by invoking the datespan.create smart Constructor on it so yeah I then just map over the uh the errors that I got from the transformation to actually return a bud request of R400 finger and then at the very end I actually do my business logic so uh creating that in the database so it doesn't actually look it doesn't actually work though so imagine I actually started a server with all the things I just wrote there and I now try to send a request to the slash confluences endpoint I sent I sent it a Json body with a name that has a lot of returning trials updates from that is not before the end and a city that also has a long name and what I get is a nice like three element array with all of the validation errors that were uh that were caught basically so all of that was based almost automatic almost because we still had somewhere to do but not much right that's the whole point and the food for a good boy request like like this so a name that is not over to inches at the eight spawn that is actually before where the stat is before the end and the city that doesn't have like too big of a name we get a two one like this uh so yeah so that's it I guess uh if you like duct tape for your books criticisms or maybe ideas uh on that link and there's code samples and Alternate Invasion versions that actually you submitted for us under this link and that's all I have to say today thank you please repeat the question for the recording was probably audience yeah yeah that's all in scratch me yeah go ahead did you hit any limitations soon enough yes uh actually like the thing I I hit some uh but like they're pretty obscure I would say let me just go back to the side so the so the question was uh uh did they hit any limitation in the scratchy meta programming um let me find out real quick uh somewhere and here I believe so yeah so for example this thing at the very bottom case X draw much error uh it's because if I just create the actual like um pattern match with just a tuple at the top right the pattern matches are generated by the by the macro are not exhaustive like ever I reported that error it got some traction but didn't like have a actual result fixed just yet so yeah so that's what that's one one of the errors that they caught of course there are some other ones that they also reported as much and I actually got fixed pretty quickly but that's like the only one that's still standing there's maybe some other ones that I didn't report because they like really crazy to reproduce I like couldn't get them to reproduce outside of this code base which is not that simple anymore so yeah that's probably my best so so you have something yeah yeah I think you can go back to me and think about one way yeah uh so when you're doing it fast but I think zero when you get one yes yes you may yeah go ahead yeah I don't uh it's just that the code before was just set up to use eithers basically so that list inside will just be always like empty and the question is uh why why do they use either of list for our free class transformation and yeah the answer is laziness basically