Building DSLS with Scala

scala.bythebay.io: Alon Muchnick, Building DSLS with Scala

Recording: scala.bythebay.io: Alon Muchnick, Building DSLS with Scala

yeah sorry from the technical stuff yes you need a bunch of software engineers to handle PowerPoint and a Mac always so actually this should be the first slide please let me know if you can hear me I'm not used to speaking with the mic like this so okay so it's four o'clock in two minutes and all of you held like six or seven lectures I guess and hopefully a big tasty lunch how many of you would like a refreshing cup of coffee right now raise your hands okay cool a lot of you everyone else okay so let's imagine for a second that we have a very own coffee machine right here in the room even better we have a barista well you can walk up to him and just request any kind of coffee that you like you'll probably go and say hi I would like a cappuccino or an espresso and the barista will say okay go out the back prepare one for you serve it to you and you walk away happy that's it well I'm here to tell you that we as developers can have the same flow the same process with our clients we can have the same easy process one magic word cappuccino with our users without colleagues and we can do that by using these cells even more once we've established this magical world cappuccino or espresso we can turn one sipping cup of coffee into a variety of cup of coffees when we walk up to the barista we didn't say I would like to have a hot cup of water with two sugars and one milk we said one word when you and the barista immediately understood each other because you had a common vocabulary so I would like to show you during the next 35 minutes that if we establish a DSL or a common vocabulary with our users with our colleagues we can enjoy the same of the benefits so let's get started now that we are all awaken from this refreshing cup of coffee let me properly introduce myself my name is alone I'm for weeks calm for almost three years now and I am developing week stalls the e-commerce platform for weeks let's get started so the official definition of the DSL is a computer language specialized to a particular application domain well you might ask yourself how is it different from any other programming language well there are two main points it's targeted at a specific problem domain and it offers a higher level of abstraction to the user why does it actually means well when you're using a DSL you don't care about the underlying implementations you don't care about data about data structures memory usage Azure cloud computing you only want to get the job done another thing that the fact that a DSL contains nothing more besides the business the business domain China salt makes you more focused with a general computer language you can model everything and anything with a DSL you can model one thing but in a much more expressive way well basically these cells are made to fit their purpose only let's see few examples of DSL so the first one is this dis cell from reform I claim this is a dis cell because we have here all the building blocks you need to build a common vocabulary we have our ingredients we have skim milk sugar brewed coffee we have our units of measurements one cup one teaspoon those combined give us a new language a new language you can build and extend and prepare more cup of coffees nothing more only coffees let's see another one another well known this a is my sequel my stick was the deal sell we all use every day to extract manipulate and analyze data we always use it when we need to solve problems in the data domain and the last example we're gonna see it CSS CSS is a DSL that allows us to control the way web components look behave and film so all of this all of these three examples I showed you have something in common they only try to solve solution problems in their specific domain and nothing additional so this is not a San Francisco bridge by the way I had no idea I'm going to be speaking here and just good luck I guess so if we take a step backwards and we try to understand the process we and they imaginary barista head basically it's the same thing we do as developers we're trying to get from the problem domain to the solution implementation we get something we need to solve and we think about how do we solve it well the problem is the road from A to B is not always that smooth sometimes it's hard and it's out because of complexity there are two types of complexity an essential complexity and an accidental complexity and if san-chan capacity an essential complexity it's the heart of the problem you're trying to solve this is the main issue trying to tackle whatever you do whichever way you choose to implement this solution it will still remain a health problem examples are sending a rocket to space building a search engine or like even supporting high log of traffic whatever you do the main stuff remains hard on the other hand accidental complexity it's something we can control it's something that we bring to our own code because of the way we choose to implement it examples focused on the complexities are writing unclear code over engineering stuff and using the wrong tool for the job so basically every time we introduce new code or we're working on a new feature or new product we are struggling with complexity and our main goal is to somehow be able to reduce the accidental complexity because you can't do anything you're gunning the essential one but you can control the accidental one and my claim is that if we by using these cells you can reduce the accidental complexity of your code you can write less code you can write clearer code and you have and you will probably have less bugs so let's see a few examples this is a test written in j-unit don't go into details just take a higher look of the test you we have some shirts equals we have some different values basically without going into details and diving in I won't be able to understand what the sisters does on the other hand we have a test written in specs to in specs to is a testing framework with an awesome DSL and look at the assertion we have hello world must out with hello hello world must end with world I can probably grow up the street to a homeless guy and ask you man do you understand what does he do and you'll probably say what the do you want from me and then he will say yeah okay yeah I know what you're trying to do so this is the concept like using a good DSL allows us to express ourselves to non technical personnel as well so we know that we've seen that what do cells are we saw a few examples and we needed language to implement these cells on so luckily for us this is the scholar conference and we can use Scala and Scala gives us some really cool features that will allow us to basically write less of a has less of like boiled print code and passed our intention in a very clear way let's see what features of Scala allows us to write a good DSL so Scala has a very flexible syntax fit for obstructions we have optional dots in method invocation we don't have to use any dots we have semicolon inferences of sugar parentheses well this is way cut off and we have type inferences we don't have to mention any type when you initiate a new value we also have concise lambda syntax which basically allows us to pass on expressive function in a very minimal syntax in one line I can iterate over at least I can sort it or I can even do them both and it's really clear what can I do it's really complicated code in a really small space another cool feature scala gives us our case classes case classes are perfect for obstruction from obstructing your domain object and your business domains while the perfect because of two things first of all we have default values we can initiate them with every Val you want and remove that hassle phone from the user we can also have named arguments we can easily pass our intention during the construction of the case class for example when I initiate name first James and les bond it's really clear what I'm trying to do James Bond also Scala has in place it's everybody I guess nobody please it's but just to make sure I'll go really quickly about the features we're going to use that your own so scallion please seeds are functions and fields that are Auto wired by the Scala compiler using the matching signature in our current scope so basically it gives us implicit conversion from type 2 type statically typed and it allows us to create lexically scoped open classes which basically mean we can extend the syntax of objects in classes that are not in our comment in our direct code base for example we can enrich scala sake and we're going to do it later on some examples for this is like an example for implicit piranha go into it basically we have a function it takes two parameters one of those are implicit and I can invoke it with say hello George and the compiler will automatically find the matching value of greeting and say hi George we have an implicit conversion from type to type so if I have a function the text only int and I'm trying to invoke into it a double I will get a compilation arrow saying well expected into actual double make sense if we're gonna add an implicit conversion from double to int we can easily later on just take a double convert it to an int and call the function thing works an example for Scala open classes I guess all of you know this right arrow right we use it every day to create like that tuples and the interesting thing in this this is not like a syntax the right arrow is not like a saved character in the scala language it's actually part of the Scala library it's inside of a fine class in the places class called arrow sack which allows us to which defines two right arrows and ugly one in the create one and basically every or every class every object you have in Scala you can initiate this methods on and it will basically allows you to take self a wherever and create it tap on with B so this is example for Scala open classes so okay we've seen what these cells are we know they were good in the day we know they reduce our complexity and we know Scala is cool so for the next 20 minutes or so hopefully less and we're gonna build our very own to yourself so what are we going to build well as I mentioned from I'm developing the e-commerce platform and I started doing that like three years ago and we have like as part of the solution we support shopping carts and shopping carts are coming over in from all the globe and every country has its own rules it's on texels it's on vitals its own calculations and we have the system that supports everything and it works and it's great and we have traits and classes and abstractions and it's nice but every once in a while when it's when are you going when I need to go back and like take a higher look at all the logic it always takes me a few minutes at least to really understand what's going on there so I thought it would be cool to spend the time we have until the end of the lecture and actually build a very simplistic version of the ship of the shipping count actually a gale based tech school matching system for shopping carts so let's say we are regardless assignment and we met with our company head or project management and it's not a technical guy and it just told us in plain English listen man I have five requirements for you for Canada the text is 50% of the count total for the UK it's 12% but when you calculate the text you should ignore the shipping cost for the US the text is 7% and when you calculate the text you should ignore the shipping cost and the discount getting complicated therefore Finland it's a really nice country it's only 5% and when you calculate the text if you leave you know the discount only if the shipping cost is larger than foreign girls and the last one well for Israel it's really expensive with 25% plus we have a special special text some tourists so we're gonna add a special Jewish text so we as good engineers god that's what we got these requirements and we immediately trying to understand how can how can we convert those English requirements to system requirements and we mainly understand that the text photo shopping cart is calculated by the shipping country text is different for each country and tax calculation is depend on other factors like shipping in discount at this point we need to stop for a second and remember the process again we start with the barista in our common vocabulary we need to establish at this point a common vocabulary between our and the rest of the project teams if we would have a common vocabulary we have to several like advantages the first one we were going to have a clear communication between all project teams I can go to the QA guy you can go to the product manager and we will all speak the same language the second one is our test we're going to speak the same language we see an emails in meetings and everything else I can actually show my test and my logicals to the domain expert and ask me man do I have any mistake here is it too complicated so let's try and break down the English requirements that you've got the forward to actual common vocabulary we see that we have all the trees you need to support we see we have M fields that will be part of the cut we're gonna calculate we have our verbs we can either ignore or add stuff and we have a predicate we would like to do something sort of thing only if a certain condition applied and the last one is we have the actual text that we need to deduct or calculate with so after we understood one of the prouds of the language let's start instantly how they call it to one with another so tax is calculate on a product crowd is shipped to a specific country and cut might or might not have a discount okay so those are all the requirements we understood all the relationship we understood it's time to get dirty and actually start programming so we start by a basic abstraction and domain model of our business objects and we're going to start by the cart the cart is really like simple representative of the actual shipping car to use two weeks but it's really really simplified we have the cart total we have shipping we have discount and we have the country please notice that tax is not a part of the crowd it's a product of the crowd calculation and we are the companion object we have shipping and discount which is just a simple function that takes a cart and returned the specific out film so far actually simple the next thing you're going to have we're going to have a list of all the countries of system supports Israel and you say Finland you can in Canada the last thing we're going to use for now is we're going to define several types and the types we are going to represent the relationship between a different object we extracted so far so we have the couch braided the couch where is the couch predicate which takes a couch and returns a boolean it's a justice signature the text calculator this is like a final goal we would like to have a cart and get the actual text from it and we have a country to tax calculation just a couple that contains a country and its current tax calculator so using those three or four objects and classes let's try and support our first requirement every DSL needs an entry point the place where we start speaking the common language the common vocabulary in my example I chose for for me this is going to be the entry point for every DSL rule I'm going to support and for it's just a simple function it's nothing special it takes on it takes in a country and it returns a country container a country container is a simple case class which takes in as in the construction of a country and it has our first verb take take is a verb you're going to use and take returns I've actually gone country to text calculation and all it does it takes a it basically it returns a tuple of a country and a function that takes on a card and it turns the simplest calculation of tax you can imagine count total times tanks so using this stuff let's try and see if we can support our first sentence from Canada the tax is 50% well we have our entry point for we're gonna pass in a country Canada we go and then invoke the take verb take method with the percentage and the final result after we remove all the ugly stuff now allows us to remove is actually for Canada take 15% let's see what's going on under the hood so for returns us a country container a country container contains the option to invoke take take take since the parameter the percentage of tax you would like to calculate and the final results is a country to text calculator which allows us to basically to calculate the tax for specific country using a shipping account so one down for the go the first part is already supported the question is can we support the second part right away well unfortunately no because by the time we finish evaluating the first power the sentence we already have our final goal the couch the text accounting calculator and we need to somehow add an intermediate step to improve the process so if you would look at the country container this is the old version we're gonna do a very simple thing we're gonna add a new player to the game called text and country container which is just another case for us who again allows us to extend our vocabulary it adds a new verb ignore now we can ignore stuff and the gross stuff is a really simple thing as well all it does it takes in a couch field that we would like to ignore and when and when it rains the calculation of the actual crowd it just deduct it from the crowd total so using this new text and country container we can now support the second language is in the second sentence again for UK take 12 and if you know shipping after removing everything looks like this pretty cool this is actual code let's see what's going on the hood so this is the same as before we get the country container we invoke take only now we get text and country container this is the intermediate step we added and now we can invoke the new verb we edit ignore 1 so we can now basically you know shipping as well and we get a final gold country to tax calculation the problem is what happened to our first sentence that we supported if you remember the final evaluation of the first sentence right now after the changes switch to text and country container but we would like to have country to tax calculation so how can we fix the first sentence without breaking the second one well this is where on Scala implicit to the rescue we saw that scallion placeit's allows us to convert from type 2 type easily without actually writing any code so we're gonna add an implicit function which all it does it takes the text in country and it returns a country to text calculation because we already have all the information we had in the first we didn't we don't need to ignore anything or add anything this is just a simple conversion from type 2 type so after adding the scallion please it function this is what happens under hood we do get the text and country as before only now because we have implicit the scylla compiler recognized that we need something else it searches and it's going on its current scope for like a conversion from type a to type B if finds that we have the function that we added and it returns the same type as the second sentence country to text calculation so let's take a short break and see the process that we've done so far we got an English requirement we broke it down two rules we understood the relationship between every object in those rules we created the domain abstractions we even read some stuff we broke some other stuff we fixed it and they repeated and we did when we repeated all the process once again so we can get to the level where all of our requirements are fulfilled right now okay let's try and solve the last three ones so now we have the self the third centers to support for us a the text is 12% and when calculating the text we should you know the shipping cost and the discount and the question is can we prove the ignore function to take multiple params obviously we can we have a very simple solution you can just use a skullet recall of our ogz and just add a small wildcard to the crowd field if you see in the olden in the new and we another additional change that we need to do is instead of just even like deducting one count field it's a sequence now so basically we're going around a fault left and accumulate all the stuff we need to do to deduct in our calculation and this is basic basically this is all we need to do to support the second sentence this is what we get but for you say take 12 ignored sound calmer shipping so I'm pretty bothered by the coma and I think we can do better and the question is how well let's try and enrich the relationships we have between the different fields on the cart so far we shipping and discount with just regular fields basically you get a cart and you return some feel of the cart let's say you do something called Cal Combinator a cult Combinator is something a bit more sophisticated which takes as a constructor a function from a cart to a field and it allows us once again to expand our vocabulary expand an old language card Combinator can support end and I need sorry so we need the end to support of third sentence and all it does is a very simple logic it takes another one it a couch Combinator takes an additional card Combinator and returns a new one when the actual calculation logic combines these two ok we need to change the ignore method to walk with the couch commuters as well it's almost the same difference we just again using common errors and we're folding left on those basically it's the same idea and now we can support the 31 we can now actually replace the comma with the world end it makes it looks much more like English okay but if we try and remove the parenthesis the Scala compiler will complain that it cannot resolve the symbol end because it doesn't know the correct order to evaluate different expressions in the sentence so a cool trick we can do is basically use the uh percent and you're just going to overload an existing operator of Scala and we're going to call the same logic that we already have in the end method and if you look at it we can now drop the parentheses and actually get the third line for you say text take zero point seven you know this and cheeping which looks really like English the question is why did the work like why does this sentence when we remove the Palestine work and the other one that in the one one did and the answer is the way Scala evaluates its operators this is actually the order by which Scala compiler evaluates every expression you can see that end is right at the top so what happens under the hood is basically that now the discount in shipping is if that is being evaluated before the ignore function is being called and we can safely get our country to text calculation as you wanted Sophie down only two to go almost there now we need to support a predicate we only want to do something if the shipping cost is larger than forward and once again every time there's a new requirement we need to find a way to extend our language extended vocabulary and end more additional abilities to it so we already have the crowd Combinator which allows us to represent relationship between different cow fields we would like to either add or ignore we already have sorry we're going to add the if the if function and notice this is a capital if because a small capital if is already reserved language in Scala and I'm going to talk about about a bit later it's not perfect and the if basically does a very simple thing it uses its get in the gates and condition and if the condition applies it returns the cow field otherwise it returns zero meaning I want deduct anything I won't do anything we also add the greater than so we can control the different logic expression we're going to pass to the if method this is like the card Combinator with everything we've done so far we started from scratch and now our vocabulary our language is richer we have there and the upper sense the if and the greater we can now add a dozens of other operators and we'll just look the same with no changes alone so after we've done that we can actually support the last sentence the first sentence and it looks like this once again under the hood we first evaluate the discount if shipping louder than 10 firm this returns us because the entire thing is composed of different cloud combinators with which you apply one on top of the another and the final result is also a card coordinator the rest of the calculation remains the same the ignore function doesn't care that we have this complex logic expression inside of it we just get the usual country to text calculation so the final rule I would like to show you and it's just how can we add custom custom lambda expression with just more freedom inside of the DSL so we said that you like for israel trail 25% tax so it's really simple this is like the basic text in Kent in country class you use so far we have the ignore function that we used so far as well and now I'm just going to add a new method I'm going to expand my vocabulary once again I'm going to add Taurus tax Taurus price and what you will do it will actually get a function from a cow to some decimal and it's going to add the decimal when you can when we're going to calculate the actual text for that cart sorry so this expression is being evaluated the same process applies and we get a county tax calculation so we've done it we've managed to support all the five requirements derived from our project manager and this is how it looks like this is actual code that compiles and works if you take a look back and compare the English version and the actual code we're just going to leave it for a few seconds so you can appreciate the beauty of it and and again if I return to the same homeless guy from the beginning and hopefully you still there and ask him do you understand what does the right side that does do you understand what's the logic every non technical personnel can see at the right side and you immediately understand what is the logic behind the calculation this was our goal to be able to express our self express our code to other non technical personnel or callings so we've worked so so hard to actually build the details let's try and use them just for the example so the first thing you need to do is an import the text D cell object let's say I put F all the color out so far inside of the text a cell object and texels are the same text roles we build from the phone the five rules so far and we have some random crowd which is being shipped to the United States and I would like again if I'm going to go with the flow to be able to say texels find rule card the problem is if you notice that find rule phone is just a scarlet sequence and it doesn't support that syntax I will get a compilation error doesn't know what it does so once again we need to use Common Pleas six canopy seats is a really powerful tool to enrich the language that we are using so we can just go and add a new PC class that will allow us to extend the regular scholar stack we all know and love and add a new verb to it find your phone is this is basically a small simple function that takes a sequence of country to tax calculations and adds another verb to it find the rule form after I've done that I can easily run the last the last line here and I will get a matching text hole for that card so we're almost done the only two things I want to talk about is arrows and exceptions we spend a lot of effort making the code look like English and make it feel really fluent but we need to remember this is still code and code always always have has exceptions and arrows and it's a good thing it's a good thing because of two things if we have done our job correctly and we managed to model every using domain abstractions and Scala case classes the compiler is actually our best friend so if you get a compilation error that means you got like real-time protection from the compiler of during you actually rightly entering the business rule or the concept that you're doing something wrong if you're trying to send a shopping cart to a place which does not exist you will get a compilation error this is like the best benefit you can get from the compiler if you model your business work correctly you will get almost in real time an indication that something goes wrong another one is we need to remember that we still have errors and exceptions and call and whatever and this is a bad a best practice everywhere but especially in these cells make sure that you're throwing meaningful business exception don't throw Stack Overflow throw I don't know like text cannot be negative something meaningful to the end-user it's really part of a DSL because you don't know who's gonna be using it so my final notes getting the syntax right is hard like if you remember I told you I had to use the capital F for the for and I used and I hate to use the capital I in the beginning and I tried really really hard to get rid of the parentheses like every time for parentheses the country and like after an hour and a half I said well screw it I understand the meaning I think it's clear enough it's a balance you have to balance between like how good does it look and who is going to use it which leads me to the next point ADSL it's only to be expressive enough for the final user like we did all this example assuming the use of the DSL is going to be some domain expert in Texas who is going to enter all the text rules by himself so we worked really really hard to make it really clear for him and really obvious what should we do but if we as developers is going to write a DSL that will allow us to auto wire eight Google cloud computing every time if something occurs we don't need to go through all the small details of a comma here and upon this is there we need to do our job we need to extract the verbs the adjectives we need to build the abstractions and we can care less and less about the syntax because we know that the end user of the DSL is some money who gonna understand the concept of it and the final note is these cells are really really fun and they're not that scary like it's really hard to get the first line going but once you do and you get the first line it's really fun to try and like see how can you extend the languages just try and do the simple thing ever a stupid DSL is my examples I didn't use Skylar macros I didn't use abstract types I didn't use a bunch of editing views like my even my polices I could have done it so much better because it's not the main point the main point is understand the concepts and try and see where the language goes to so I encourage you all goal and white and write your very first simple stupid DSL and have fun with it that's all [Applause] that generated this automatically oh sorry sorry the question was if I answer correctly is there something that can help us generate the communication regarding these cells yeah and we worked in TDD in weeks so like every answer I get like is the test out the answer like just look at the test and you will see like all these charts and what you can do what you cannot do basically and if you haven't done it but I guess if you have to give an example to and non-technical personnel I would just I guess you have to write it down I'm not familiar with something I'm sorry I told you basically like what do you get from if it's statically typed language then the benefit that you get is the compilation stuff that you get that you know something is wrong during the compile time but like if you try and do a DSL I actually have a version of the same lecture in the typescript okay typescript so it looks pretty much the same only without the cool stuff of Scour implicit which is very very cool like for the second example where I can just told you listen I'm going to fix it using implicit and it's like magic it works I actually had to add an additional disgusting function called I am just converting ink so it will work because I had no other option and in Scala I can just use it and it puts it and plus the syntax stuff if you can ignore dots if you can you know the parentheses like all the options you get like the clearing the eye those are two benefits I see from Scala implicit and the syntax yeah okay thank you guys for a good time [Applause]