Random Data Generation with ScalaCheck

scala.bythebay.io: Daniela Sfregola, Random Data Generation with ScalaCheck

Recording: scala.bythebay.io: Daniela Sfregola, Random Data Generation with ScalaCheck

you so today I'm going to talk you about random data generation with color check it's not actually a token skaara check so apologies if you have been a little bit misleading what I'm going to do today is actually I'm going to present a slightly different way of using scholar check so usually you know when we test things in Scala we have two approaches the pretty simple approach of just using predefined test data and then use that taste data to run our tests and we have the amazing scalar check library that will do all the magic for us so what color check will do is that it will automatically generate some test data starting from the most generic case scenario and it will shrink it until you will find the data that breaks our test um and these two approaches are are a really common I believe that everyone is aware of them so just review really quickly what they are again static features is basically you predefined or your test data and that is ok but that means that your test data will be completely biased so how many times you guys well it happened to me that a test failed sorry it happened to me that a band was found because I wasn't testing that specific education REO and then you have scholar check so really quickly scholar check is a really powerful library that allows you to define certain properties over your test data and what it will do is that it will generate data any will find an example that breaks your test so is really really cool but there is a problem the problem is that wine approach is completely by you and the other one doesn't work well with side effects so for example i'm currently working with a retailer with the startup that works in the retailer sector there is called pay touch and we have this service it's really simple nothing fancy we have a service that we call core that has a database that is dedicated to the service and then we have a recipe I and obviously that there are some integration with other parts of the system and other third parties and we really really wanted to test this really well because it is the core of our system and we are set up so we do need to be reliable so what we said is ok we are going to do BDD we're basically each test will define the behavior of our service and that work well but at the same time we had because the retired business is so complex we couldn't really think of all the possible edge case in re so we wanted to do something a little bit more than testing with predefined test data so we thought okay we're going to use car check but then we realized that we had so many tests that Skylar check wasn't really the case the the reason why Skylar check wasn't useful for us is because we are using postgres so we were testing against the real death race that means that if we have 600 tests skull attack scarlet scarlet check will run each test 100 x 600 x 100 that sixty thousand tests and the database wasn't really happy about it also the test we're taking right now the test take around four minutes so um we could have improve our infrastructure to have a bigger database to end all the load but again we are a start-up we're not Netflix so we don't have a lot of money to spend on infrastructure so we are in a situation where basically we have the normal approach there is a little bit simplistic we want to make sure that we represent all our education areas but at the same time we cannot really use color check so what do you do and there is the reason why we thought maybe there is a way to run the test once but using random data so this way all we have to do is to measure the side effects that our API generates that means that every time we make a call we will check that the data in the database is is what we expect that we make the call to the two parties and so on and so forth so that's the reason why we spend the squid some time we played a little bit around with things and we created this library there is called random data generator and so far is working really well for us we are going to see later on how we actually implemented it is surprisingly simple but let me show you really quickly what these APR these library looks like again really simple so suppose that you have a case class example with the text and an end all you have to do is just just say give me a random instance of example and we will reduce color check to do all the work for us so how do we use these in a test again nothing fancy so this is a normal test that probably you guys have seen around so we have a test that will assert that we create a user so we have a random user and no matter what the user actually is we want to check that when we call the api post users then the API will respond with 200 sorry 201 created and we will have to check that the user is actually in the database what you would normally do is that you will pre define what a user is what we have done here is just wish I actually just use a random user so that's pretty simple to use you don't have user name is Kayla check you have to rethink the way you are testing things yeah you really know you really don't you just have to change that line but also obviously this means that our tests are not deterministic anymore right because every time we run our tests it will be run towards different test data so obviously once we find a session a set of data that breaks our test we want to have the opportunity to replicate that specific session so what we have done is that every time you run your tests we will select the seed and that seed will be used to generate all the random test data so if you find a bug obviously you want to fix it so you have the opportunity of fixing the seed rerun the test again this time the test will be deterministic and you will have an opportunity of fixing whatever bug it's in your code so we start using this approach and an amazing start working very well for us in particular we we found out that we didn't worry too much about all the edge case scenarios that we didn't cover because sooner or later this library was our lighting issues in our code and yet we had the opportunity of making this test deterministic when there were issues and amazingly we found so many bugs all over the place so that's amazing right because we found the bugs it wasn't the client so that they're actually pretty cool so we keep using it again again and again and we also found out that the code was actually a lot easier to maintain the reason why is that is because the test doesn't really care what user is because it's going to be the library that will generate an instance of a user so if I suddenly change the user and I had the field in my case glass the test will be exactly the same so there was a another nice advantage as well also our tests become a lot more readable so when you have tests that highlight Aspasia I light a specific issue in the data like for example this test is a test that will make sure that if you try to create a user that has an age that is below 18 or 21 since we are the state um it will it will reject the request and it will make sure that we didn't actually create the user in the database so in this code is really clear that the important piece of information of our case glass is actually the age we don't care about anything else the important bit is the age but how did we implemented this library this library is mainly composed by two elements the first element is obviously scholar check right we don't want to reinvent the wheel we have this amazing library that will generate the data right and you will generate the data in a way that the HK scenarios are actually more common than the regular case scenarios so we will select the seed and then give an arbitrary we will get the generator and we will get an instance of tea okay little bit slower arbitrary arbitrary is the class of skull attack that will include the generator in your scope that's all it does nothing else the generator jen is the heart of skull attack so that's the real magic is just is going to be the class that defines how you generate a specific type and then obviously we can sample it by giving it a set so we we have two entry points right so we have the seed and we said that we are going to select one for you and we have the arbitrary but where is the arbitrary coming from and that the the amazing thing about Scarlett check shapeless I strongly suggest you to use it even when you use color check or in general when you need to generate things automatically because basically what you usually have to do with skull attack is that you manually have to define all your generators and that's pouring in one of the boring so Scott check shapeless actually will automatically create all the generators for you and way it works is that it will be able to infer automatically the arbitrary as long as T is an algebraic data type but for humans that means it's your case glass or silt right so it's a really really nice library again that I suggest everyone to check out but obviously the compiler is doing a lot of things a lot of things that we don't want to do so hope obviously the the completion time would be affected by that but hopefully miles will fix that as well so things are going to be amazing in the future just let him work so what happens if Scarlett check shapeless is not really able to infer stuff in pariwar type obviously we still have the old way the boring way so we can still define arbitrary is manually so for example if we use Java looted currency there is a common way of defining currencies all we have to do is to define an arbitrary and say Darren arbitrary is a generator of one of the available currencies so it's not it's not too bad so once you start using this library you will find yourself arguing about your tests so what happens is that you will either think that your code is wrong or the data that you generated is nonsense so when when you use this library and Skylar check in general you do need to make sure that the data that is going to generate make sense so what happens if in your application you have particular requirements for example when you generate a string with fellow check this thing over there is a string it's surprisingly but it is a valid string so maybe in particular if you have a database that is not a valid string for your contacts so what you want to do is that you want to tell Skyler check what is the valley string in your context so what you can do is to define a new arbitrary that is just saying that a string for you is just a list of letters but what if you you are happy with the way a type is generally defined but you're not happy of how that type is defined within a case class you can still do it so for example we have this case class there is a person with the name in an age and we we just define that an arbitrary is is composed by a name that is one of the yellow John marching and an edge there is an inch between 0 and 100 so that there is a little bit of configuration still to do ninety percent Ori Ori ninety-five percent of the time scholar check will be enough but obviously you do need to be aware of making sure that your data is reasonable so actually you know Skylar check is not a library for a random data generation and this is really weird because my talk is actually called exactly the hurry and the library is actually told is actually called random data generation so Skylar check is not a library for generating random data the reason why is that because it does color check is biased towards edge cases so if you use this library to like populate the database or do some statistics analysis or whatever you will find that most of the data that is generated covers the HK scenarios rather than the more standard ones so if you are looking for a tool that will generate random data to populate the database this is probably not the tool for you but if you're looking for a tool that is generating data in in a way that he will make sure to test the education areas first that is the sole for you so to summarize most of the time Skylar check will be enough for you and I do strongly suggest you use color check if you can if you cannot really use color check because you have a lot of side effects and that is stopping you from doing that then this library is a good compromise between completely using predefined test data also be aware of what it means to have what is the meaning behind your data because otherwise you will find a lot of tests that are failing simply because the generated data doesn't really make sense also um if you use this library you will find out that every once in a while test will fail randomly and this is the nature of the library so obviously do not ignore those test failures there's a whole point of the library just take the seed number reproduce the session and trying to understand why the test failed and again please be aware that all the data that is generated will look weird so don't use this library to generate data to maybe story another hobbies or or stuff like that please just use a for test purposes and then last one is just a great sentence from a guy did you might probably have heard of a tie astre they just remind us why testing is so damn difficult and he's basically saying that we testing you kind of really demonstrate that something worked you can only demonstrate that something doesn't work and there is the reason why testing is sometimes so damn difficult so that is everything for me if you guys are interested I have a look at the library the link is there and that's about it thank you very much guys are there any questions so we have x question yeah vision is you have your service and your test was successful you're our best twice and successful so at what point you'll make decisions it actually some crazy production sorry can you repeat the question yes [Music] well there's the whole point of testing you cannot prove that something is correct you can only highlight presence of parts so obviously the more you run the test this is not Skylar check right so if you want to have a higher confidence that the test will work then Skylar check is the way but obviously the more you use this library the more he shoes you will alight so know if the test pass doesn't mean that the code is perfect just means that i didn't find any problems with the tests theta that he generated oh you got wrong there is actually there is actually a nice higher order function in Scala check that will deal with that so once you generate for example a generator of a string scholar check as and higher-order function that will automatically convert that generator or string into a generation of an option of a string so that's that's easy enough to do you're answering a question was more like you're you're a user it has attribute I feel each time employment want to test that user with none and also that using self so you need to choose voice in that case I will probably due to tests because using random test data means that you cannot really predict that that field would be some or you will be none in that case I will write to test saying these feel needs to be done because this field needs to be some because in that case I think it could be summarized to this case where a specific field within the data has specific meaning so in that case yeah I wouldn't leave it to chance because again random data means there can be some it could be none you don't know so I will keep it separate see you just ring specifically sometimes you'll get a few strings Spence won't you do you need to write two versions of every test to test for say empty string cases and other like zero furnitures um well if you have a specific requirement that you should never have an empty string you should have a test that says okay if I send an empty string my behavior will be the one that I expect if your context of the system will guarantee that you will never have an empty string what you can do is that you can change the generator of a string to not generate empty strings cool all right thank you guys very much yes