Scale By The Bay 2018 Unconference: Omer Van Kloeten

Scale By The Bay 2018 Unconference: Omer Van Kloeten

Recording: Scale By The Bay 2018 Unconference: Omer Van Kloeten

you hi okay so hi I'm over I'm a staff engineer at the Tel Aviv office of we work and I just decided that I want to do a talk it's nice but I don't have that much time and this talk was originally 40 minutes so I'm doing to do the totally rest edition while preparing for it I noticed that I had signed up first track on my birthday so this is the birthday edition yay there's no cake I'm sorry so let's start with a good like something optimistic everything is terrible we are human and as humans we make mistakes we make we have errors to err is human we have typos we have bugs we have aggressions things that are ring you're ruining our days and this is a graph I want to show this is the cost to fix a bug over the lifetime of a system from requirements to design so on to operations so two things I want unit to notice here is one that this is a logarithmic scale so this is logarithmically more costly over time and the second thing I want you to notice is this is for 1981 so before I was born so we fixed it right because Unitas the unit testing and unit testing is awesome and we have a way of finding problems really early and refactoring our code with confidence there's also it's a good way to document the usage of our code right but unit testing is less awesome it is much more code to write and more code to review and more code to maintain and so much more code there's systems where it's six times as much code in unit tests as there is in production code and is it exhaustive really because 100% coverage is a lie and it doesn't really mean that your system is actually covered by tests it just means that you have written code that goes through all the paths in Huawei our leather tests our poor documentation reading code is hard through it every day so writing tests which are code and having people read them for documentation is a pain another thing we have to deal with is the Virgen behavior we have documentation let's see this mmm this function connects a cat to all veterinarian clinics ok so we have a cat ID and return the number of cat clinics that we've connected it to makes sense right but now the code looks kind of different there's a cat ID and now there's a clinic ID and it returns a boolean so what does it even do no no looks at let's look at the tests right a cat may not be associated with more than one clinic what I am not really sure what's happening in there's exceptions that are not documented this is a good specimen so what if we could have compiled time annotations that enforce invariant business rules and are a good replacement for the basic validations and the tests and the documentation and just like put it there and you can be impact our code our codes users and we can discover failures really early and so on and so on types right just make sense this is call a conference I think don't think anyone would say anything else right so let's take a first step this is a type it's connection it accepts anything that's a host anything that support who thinks this is a good example who thinks this is not a good example unanimous across everyone here in the room you're wrong because that's a maybe it might be a good thing because Kent Beck describes the lifetime of a system as exploring expanding and extracting when you're exploring you're actually not sure what you even supposed to be doing the domain has not been flushed out yet what it even is a host man I don't know why would I encode it into my code it's gonna change I know it's gonna change there's nothing there's no invariance here when you're expanding you have to go fast you've gotten traction in your system things are moving quickly there's a lot of crashes in production but the business is pressuring you that you have to scale up you have to build new features you have to do everything there's no time to encode invariance because there are no invariants they are still changing constantly and then you get two growing teams where this is a bad things now we're sorry to extract with this is a bad thing now and you're growing your team and you're refactoring you're feeling the pain where none of your invariants are encoded into the system so you're building new things based on shaky foundations so let's go back to this again and let's constrain the domain so we have any sure that's awesome but let's constrain that to an eval where we can have minus 2.0 that's not a good port let's have integers yes that's a good way to represent a port right nice now we have this but we have to have lots of documentation because it's not really constraining enough right I can send a minus two to the port and just get an exception right from the constructor because that's not a valley port and I get an illegal argument exception this is not fun so can we constrain the domain even further can I just say valid ports period that means I wouldn't have to have the contagion that means I wouldn't have exceptions right I would just declare that I want valid ports so I can use refine types who knows refined types okay cool so refined types is the library where I can say things like port is an integer refined to be only in the int close interval between 0 and 65 64 K that means that only the types don't the instances there are allowed are ones that are the fall into that range and our integers so my connection class now looks like String refined to be ipv4 and a port that support right so this would fail to compile it's not a host it's negative one and this would compile just fine because it fits at compile time and compiler checks us to make sure that we're good but that's it cheating because we don't use literals we use values that we get maybe the int maybe it's maybe a port so we use refine V which is a macro that gives us either a string which is the air that we got or it gives us a for it so now we have done this explicitly we know what failed why it failed when it failed and we are proactive we are trying to figure out what is happening before just chucking it onto the implementation and telling it hey tell me if anything is wrong so refined types are here and github whatever they are type level combined able predicates so you can write things like I want a list of integers that's refined to be both non-empty and every single item on the list has to be positive that's nice so we have all these combiners there's an entire ecosystem of libraries that have refinements for their types so you can just add that layer on top of that and have compiled them safety safety there's a ton of things I am NOT gonna go over this slide nor have I ever so we've achieved something really cool we've taken runtime exceptions and pull them back in time in our software development lifecycle all the way to compile them errors we've obligated our callers explicitly to match some sort of invariant we've documented our code because we've said I need a port what is support that's support so it's easier to understand even if you don't read the documentation and we've written less code we don't have to write the tests to make sure that the port is a port because those tests won't even compile vadym set on themselves and we've done it automatically using compilers and macros awesome we have an exercise to see how we can use that newfound understanding but no there's no time to do it so next interlude what can't use statically type you can't stand eclis type IO barriers anything outside of our type systems cannot be typed I can't type the network I can type a database I can type a file system or the OS calls themselves because they are fundamentally on type they're outside my type systems how do I solve that yeah better abstractions better libraries that give me better types and handle those cases in the correct fashion and share that with me through the type system as much as possible there are a ton of them another thing I want to say is the tests are never going away ever sorry so types are really good for validations like port is a valid port number cool but tests are still gonna be there for the market more complex business rules it means we're gonna write less tests but we're still gonna write tests tests are good so just in case anyone was thinking otherwise exercise number two because we didn't have the first one we'll have this one so let's take or we have a way to pet dogs right I send them a dog ID I pet the dog but if I try to set a cat ID I get a byte exception because you don't pet cats the same way you pet dogs so what do I do ok let's write as this this test says cats should not be padded like dogs too late our users have already used this API they don't care about our test they have never read our tests they don't care about our code at all they have an integer they sent an integer ok so what do we do we can use something that's like ID types cool so I have a dog Eddie that wraps an idea it's a value types so we have lower impact and we have a dog a DNR class we pet the dog lady and now we just can't send a cat ID into the function that is awesome but if I need to send NIAID any ID to something I can't because it's not an integer so I would have to do dog ID ID which is annoying right it would make sense to send to make sure that the dog ID is an integer but an integer is not a dog ID in implicitly right so we write this cool we have extract ID it's implicit with scope a dog ID is now an ID implicitly but now we have a ton more animals and we have to write one for each of them or complicate our domain model really really so another thing that's not so much fun we can accidentally create cat IDs right we could just create a cat ID I don't want that I want cat IDs to be created by the persistence layer right so I mark the oh sorry about that what's going on okay so I mark the constructor as private cool but I can still actually accidentally create well accidentally create an instance of cat ID of the of the cat with a dog ID because I am unpacking the doggie explicitly writing it as an instance of candy not so much fun so what can we do about that we can use the sealed abstract trick who here it doesn't know the sailed abstract trick okay cool so sealed means I can only extend this class inside the same file right only implementations inside the same file I can extend this type abstract means I can create an instances of this without extending it that means that only the own way to create an instance of it is to write a function that creates it inside the same class right so what I do I add an employee function on the companion object and I mark its access to only be used by the persistence layer and it creates a new instance of cat ID an extension of the cat ID from class that's nice I can add business logic on that I can validate things make sure that everything is in order but I can't because you can't extend a value class so that doesn't work ok we've hit at that end so either I leave the an eval out or I don't do this ok so let's look at another thing something else right let's look at tagging so tagging is just telling the compiler I'm adding and something empty on top of this type I'm saying this is an int with a dog ID so this okay tagging so what we do is we tell the compiler here I know better than you and I'm telling you that this 10 is an integer with a dog ID so that's ok cool um fine you do you but now I can use green I can give an int with a log ID and that means only types of this and instances of this this type this tag type can ever be accepted so this is a nice thing nice thing because int with doggy D is still an int so I still got the value of being able to unpack that into primitive there's zero runtime overhead and accidents are far less likely because as this is of is an anti-pattern in scala there's a convention for this there's int at at dog ID that's tagging please don't roll your own there are way too many implementations of that shapeless Cal Zed and so on have their own but this is cumbersome so let's look at something else there is something called new type in Haskell and that's just it looks like this and it produces this it's a macro I don't want to go into this way too much and it looks like a case glass right but it's not a case less it's actually a tag what it is it's a zero cost wrapper that just adds a tag on to type looks like a new type and I guess new type it's coming in Scala - at Scala as well - his way Oh pay as opaque types you can read the SI P and it's really cool so let's look at our cases all right we need to send an integer right an ID send me your ID it's not an ID so we have to work really hard to get to this point we just change the annotation to new subtype now it's an int done cool oh okay so what do we do with our explicit logic and making sure nobody instantiates it like outside the scope and we can do that by just making it not a case class just the class and now we can do this easy okay so a recap of that simple is this is what it looks like now we have a doll ID which is a doggy this is looks like just a case class right and as there there's zero overhead and unwrapping it is just reference again that's it but we need to go deeper right let's look at persistence we have Corinne we have deleting and we have updating all of those needs IDs but creating doesn't immediately know it doesn't need an ID so we don't want to encode the ID into the type because IDs are implementation detail there a way for a persistence layer to understand we're referencing to the correct part of the object to the correct object in it so we can separate the concerns out we can have a final case class that's a dog with ID so we da-ad we have a dog okay that's cool and then our functions look much cleaner right our query just gets an ID our create just get it gets a dog or update gets both and we get back what we expect okay let's make it generic awesome now we have a whip with entity and ID so now I can just say every entity has its ID we can couple them together we can decouple them at will the reason I wrote it this way is so I can use this the infix notation which reads nicer its entity with ID right so that's nice for everyone and now because we are not doing functional programming to the extreme we're gonna wrap that with with a repository and have a repository of entity and ie and we can just infer get all the types for every function there you can read more on this nice little blog post okay but we can create a repository of a dog and cat ID bummer what have we had if we had typed connections between the dog and a dog eat dog ID right if I could write something like dog ID represents dog that would be awesome because I just wouldn't be able to create repositories with the wrong types so what I do I take our example and I just tack on implicit so now I have sorry and implicit so now I have a represents type it's gonna be essentially a trait that when when I have an instance of it with both types that means I have evidence that the type represents the other type so now I have implicit let's fix that so now I have if I have an implicit value of ID ID of dog which is dog ID represents dog now I have a typed connection between the two types which means that I cannot create things like dog with cat ID cat with dog lady or a dog with doggy D then query it's for cat ID nice recap next so what's to take home on this really simple there one there's a lot to learn it's a photo opportunity so you're welcome to take a photo there's a ton of kind of types and scholar path-dependent types that we didn't go over essential types that part of others are going away so be careful but first before you start using your types do one thing change your build make it more robust - axe fatal warnings I love it someone said I don't remember who Fela warnings is like global warming you keep saying you need to do something about it but you haven't and now it's too late so please start early people will hate you in to begin with and then love you later use linters here's a various number of limiters there is a talk I gave about powering up your build that's there's a link here I'm gonna release the slide sire it's about how to make your builds more robust summary yes we've done all these things and now you need to think about where you benefit that's it thank you [Applause]