The Essence of Testing

SF Scala: Bill Venners, The Essence of Testing

Recording: SF Scala: Bill Venners, The Essence of Testing

I'm bill Venters and I ran up here to change the the title here cuz I did just give this in London last Friday I told Alexi that it would be practice for tonight so you guys you know it's not you're not getting this cold I have actually done it once and what what I want to talk about was what I have learned over time doing Scala test essentially and it all goes back to this person anybody who that is anybody recognize him this is the person that first introduced object-oriented programming to me was a younger version he looked a lot younger back then it was way back in I think when it was the late 80s I think I was at a company in Santa Clara Kaylie instruments and we were all C programmers and we wanted to use this new language called C++ we decided to use that in a project so they enola new object under programming or C++ so they brought in some expert from the outside so-called expert and it was a guy named bertrand meyer who is a french guy who designed i phone and he wrote this book which was really seen as one of the major books in sort of defining what object on your programming meant and so he he actually talked me you know first time I heard about subtyping or inheritance and that sort of stuff in that course and it was you know for him I think he was you know I was of course on C++ but he talked a lot about Eiffel because he thinks it's better right so he you know he his his languages and sort of take off as a mainstream language but if you look at this book I reread it parts of it recently just to prepare for this talk and I was amazed at how many things he sort of figured out so a lot of what he he figured out did actually get adopted but one of them that he he he did sort of define was this thing called design by contract how many people have heard of designed by contract you heard that yeah okay so you've heard of it came from this guy Bertrand Meyer and what what he said is he's trying to figure out how to define that software's correct and what he said is this is how you decide if software is correct ba Q and what what P is is it's an assertion and Q as an assertion so he also sort of defined what assertion meant and fleshed out what that meant in that it and this thing called an exception gets thrown if the assertion fails he sort of like to find that and what P is is is a kind of assertion called a precondition and Q is a kind of assertion called a post condition and then a is an action so it's like something you do so if you call a method there is a precondition assertion then there's an action and then there's a post condition assertion so an example of it is the precondition might be X is greater than equal to nine and then an action would be add five to X and then a post condition assertion is X is less than equal to 13 so anybody know what nine plus five is it's actually not thirteen fourteen yeah so this actually isn't I mean if you wanted to stick with nine as your precondition you could change this to 14 and it would still be correct right so if X greater than equal to 14 is the post condition that says that there's one more value of C is that one more value is allowed X greater than equal to 13 allows 13 X greater than equal to 14 doesn't allow 13 so that means there's one less value and that makes it stronger right so you can strengthen an assertion or weaken it so over here if you stick with 13 you could actually make that X greater than equal to eight right so X greater than equal to eight means that eight works now but it didn't before so that's one more that's one more value so it's weaker is that right actually got this completely wrong in London in other words uh if that if it allows more values then it's a assertion it's easier to achieve if it allows fewer values it's a stronger assertion it's hard to achieve right that's his idea and one of the ideas of designed by contract is that if if on a precondition is weaker it's easier to for the client the person calling the method it but it's it's harder for the person providing the method because they have more to worry about and so one person's obligation is it is another person's benefit so design by contract was was the idea that there's there's two parties to every method there's the person who writes the method who provides it and there's the person who calls the method who uses it and an obligation for one is a benefit for the other so if if the post condition is strong then that's harder for the person writing the method to achieve so it's obligation but it's less to worry about for the caller right so one person's obligation is another person's benefit and that's it's a fundamental to design by contract is the idea that it's deciding who is that who's responsible for what right so the the preconditions are the responsibility of the caller that's their job to get that right and the post conditions are the responsibility of the the person providing the method okay so that's B a Q and if you take that to the limit false is the hardest to achieve assertion so that's the strongest one and true is the easiest to achieve assertion you'll have to do anything anything's true if the assertion is true I mean that that's what it is then you don't have to do anything to achieve it right okay so the deal with assertions is in Eiffel so this is the eiffel programming language which is was you know Bertrand Meyers language because he's French he named it after the Eiffel Tower just look like this they're boolean expressions so N greater than zero means that someone's passing in in it has to be greater than 0 and then X does not equal void is like saying X is not null and then a semicolon separates them so in a semicolon is seep and separated just like in Java Scala but in in the context of assertions it was an implicit and so all the little pieces of the individual parts of the precondition are ANDed together they all have to be true for the 4p to be true right to be a past and so one of the things that he he had in eyeful years ago before it was cool is semicolon inference so just like in Scala if you have a return there you can leave off the semicolon and then but these are still implicitly ANDed together it has to be both greater than zero and access to not people not be null and then in his and Eiffel you could label these and the reason is that what happens when an assertion fails when it's false is an exception is thrown at runtime and so you get a stack trace but if there's a whole bunch of assertion let's say there's ten of them you don't know which one failed unless you have a little note saying oh by the way an was not greater than zero or the non positive one failed so that's that is what positive is force not so much to say by the way in and greater than zero means positive because that's kind of obvious you can just see in greater than zero and know that it's more when that fails that was this part of the thing not the rest okay so that was a in I fold back in those days and this is how a function or a method looked he didn't call the methods but the same thing in Eiffel he's got a name up there square root takes a real number which is like a double and the result is double and that look at that that looks a lot like Scala there's a colon and Anna type 3 and then is is like the equal sign this is now this is the body but he always had three parts he had a require section which was the precondition he had a do dot dot dot which is the the action the a and then there's an ensure clause which is the post condition so he really formalized that in the language that you actually have to write those those three pieces when you make a function or method and so here it's a square root so the X has to be greater than 0 or equal to zero because because for a square root you can take the square root of a negative number right so that's your precondition and that's the responsibility of the caller the client and then the dot dot dot is left as an exercise for the reader because it wouldn't fit on the slide and I don't know actually how to implement that and then there's the insurer and here because it's actually a floating point math it can't be exact so what he does is check for correctness within a tolerance so if you if you take the result which is the candidate square root that's been computed by the do part and you multiply it by itself so you square it and if you subtract the original it in math it would just be zero if it's correct right that's all you need to check but in in in in a floating point math it might be a rounding slightly different right so what he does is he takes the absolute value to make sure that that rounding difference is positive and then he just make sure it's within some tolerance that he cares about however that's defined and that's his post condition and so that was how you write a method in in Eiffel you have to have the P and the a and the Q okay so so essentially the idea of this contract is that that it's the the idea of there's two different roles and these are people's roles the person who call who writes the meth and the person who calls the method so if I'm writing the square root method then I'm providing it to you and so what I promise you know my the contract or the deal between us is that if you give me good data and I'll give you good data essentially if you give me a number greater than or equal to zero something that passes the precondition then I promise to give you something that approximates the square root of that number within a tolerance and that that the the actual specification of what that method does is is this require and ensure Clause sort of actually specify that that's the specification of if the method actually does that then it is correct sort of specifies what it has to do and it can test that it does it right so another thing that he had in in it sort of fleshed out back that way back then was that assertions are you know first so they they throw an exception so it's not something you want to actually get at runtime and it may take time to do this computation here am i squaring something and subtracting and maybe won't take that much time but it might take some time so you could turn them off in Eiffel in a very fine-grained way so in a class-by-class basis you could control which assertions execute and don't so you can say they run everything when I'm developing and testing but turn off certain ones when I'm in production once I'm confident is gonna work and the default in Eiffel was that if you didn't say anything though the default you would get is is the insure clause would not execute and the require clauses would execute I thought was the default so you check your preconditions always but you don't check ever post-conditions okay so there was one other well it was actually more than one kind of assertion that he talks about the main other one was called in an invariant and so it this is if you just look at functional like just functions pure functional programming you'd have to worry about a variance because you just have functions and that precondition and post condition is enough but in a in an optional program you have a class right and a class has some data in it usually and so that data may have requirements that like one of the variables is always twice the other one or one of them is the Roman numeral version of the other one or something like that so if that should always be true something that should always be true about the data of the object is called an invariant and he had a way to express that and eiffel also and then he would check that he would add them to the preconditions and the post conditions of any public method so to the way that in eiffel a method was deemed correct is if the preconditions in a variants succeed then after you execute the body that the postconditions invariants must succeed and if it does then it's correct so that's like a definition of what software Krakus means when you start doing functional programming and up you know like Scala has a lot of immutable objects you wouldn't really have to worry about the invariants except at the end of the constructor so once you construct an object to guarantee that it's in a valid state to start with you check the invariants but after that it can't change so you don't need to worry about after that so it'd just be pre you know p AQ after that so okay so you can actually do this kind of thing in Scala if you want the design by contract things I want to give you a little demo of that I mean first of all what I what I decided to do is write a square root since he had this is like that stuff came from his book from his examples what Scala has let me just demo Scala square root which just it just calls the Java one if you say math that I just started this rebel so I think I have some Scala test stuff in it but it's just math it's called that math that square root of 9.0 you know you take a double number it's 3.0 right or if I say a 4 I get square root of 4 is well that's - let's do this creative - that's a more interesting there you go there's an interesting one so that's an approximation right probably goes on a little longer than that if you pass a negative number you there is no such thing as a negative the squared of a negative number what this guy gives you is not a number so what I want to do is write a square root that throws illegal argument exception right so that's the difference so my square root function takes a double and it says assume X is greater than equal to 0 and X is non positive so I'm going to show you what happens when I take the square root of infinity anybody know what happens if I take the square root of infinity and that's called double dot positive infinity you know what I get yes that's right the squared of infinii is obviously infinity so maybe that's correct I don't know but I kind of don't like it it seems like I've overflowed so I'm gonna actually disallow that in my method also so this assume Clause is my precondition and assume is a method in Scala Prieta in pre death with a predefined check that's imported implicitly this does exactly the same behavior as assert but in it is intended for preconditions especially so I'm assuming that this is true in my method is what that says and so I'm a demo assume so I'm gonna have to say cuz I've imported Scala tests I'm going to have to say pre def datas I'll do assert first this makes a value x equals 33 plus pre diff dot assert certain X is let's say something that will work X is greater than 0 so what assert did does is it returns the unit value to mean pass right it worked that's indicates success so if you have something that fails it throws assertion error right assume does the exact same thing but it it gives a slightly different error message if it failed instead of saying assertion failed to say his assumption failed but it's an assertion error and both of these just like Bertrand Meyer sort of defined in in design by contract and Eiffel can be alighted at runtime so you can set in allied level when you compile Scala code and it won't actually run those right so that's that's the design of a contract idea and then this ensuring thing is also in pre de and the way that works is you call it on something so math dot square root of x basically if I pass my precondition I can just return the result of calling math square root that's what I want to return the whole point of this function is just to like throw an exception on those preconditions otherwise I okay I'm happy with what math square root says but just to sort of do the precondition I say ensuring and what that is is implicitly added to everything so I can call ensuring on anything so I'm calling it on the result and then what insuring does is it passes the results into this function which is the next thing you put and that's where you do your assertion so what I did is I use this you up ULP gives you the distance between the dis double and the next one and I use that as the tolerance check and so then I just do his is the same thing you use square res you subtract to actually take the absolute value make which makes it positive and you make sure it's less doing before the tolerance and that actually passes so let me demo ensuring real quick I can say X X is 33 right so I could say X ensuring and then there you know the thing basically 33 is gonna be passed in here and so I could just say if X is greater than 0 that's going to pass my assertion so instead of 3 returning the unit value here it actually returns what you called insuring on so that it will be returned from the method right because right here the last thing that's going to happen is the result of ensuring and that's what gets returned so if the the assertion passes you want to return whatever and trick been showing was called on right so if you call ensuring on on 22 then the whole thing returns 22 if the assertion passes but if it doesn't pass then you get in there search an error so this can also be alighted except the function call doesn't actually get delighted that she does call that function but on the inside it doesn't do anything it just always passes so this by the way was something we added when we were writing the first addition because it didn't exist and we were talking about designed by contract in the book and so there was an assumed but there was no ensuring so Martin invented one and then we wrote about it and then there was a bit of as one of the few times that we had a bit of a disagreement Lex spoon the other author and I sort of were debating if you use the design by contract way the assumed way of doing preconditions or to do the job away because in Java it's a little bit different Java always would throw an illegal argument exception they would not alight it you couldn't like you light it and you didn't throw it in air for that like a precondition check in Java was kind of part of the contract if you give me a negative number to my square root I will throw illegal argument exception in this case that's what I'm doing right so that's we also added require at that time which looks like this and it's the same behavior except it doesn't throw assertion error and it can't be alighted if there was illegal argument exception so if I say instead of a sir assume like I say require I get requirement failed right so here one of the things that I did in scholastic which the Galactic is a library came out of kind of grew out of Scala tests there were things people asked for in Scala tests that they also asked for that they could use it in their production code a few things so I made this sister library that is released at the same time called scholastic and so one of the things I put in there was requirements because if you use galactic requirements you give the error message basically you don't ever want a requirement to fail because you've just you know something blew up at runtime and something that's a you basically discovered a bug but if it does you want a nice error message because that's like forensic Clues that it's in the log file that helps the you know human has to come and try to figure out what we're wrong and fix a bug in the code if that happens right so if I use the Galactic require then you actually get 33 was not less than zero the same kind of error message you get when an assertion fails in the scala test okay so so that's require and what I think no there's a lot of things in oh yes sorry question yes yeah the question was can you catch an error and log it out rather than blow up yeah you can with a catch clause and then yeah we can do that no it's not built into Scala of events calls on the JVM and the JVM does I think allow you to like define as a configuration the default catch for a thread maybe that might be a true but that's on the JVM that's not really Scala I don't really what you do is like if you're a a web framework and a request comes in and then whatever that does blows up you don't actually just die you catch it and you log it out and then you continue up with the next request so okay so anyway I think when I when I met bertrand meyer he was a little bit frustrated because he had decided he had just done this amazing sort of intellectual he came up with all these things and people didn't use eiffel right they used c++ and I think it's like oh so when I was reading that his book again just to prepare for this talk for London I was like wow you know he actually sort of figured out a lot of things before they were popular like he talked about the the assertions is how you specify the software and that was way before BDD it's like hey tests or specifications and even talked about maybe you write the specification first way before TDD was like going around saying hey write your tests first so he he I think came up with a lot of stuff that maybe he doesn't get credit for so maybe he should but one of the things that I think for to fit in that category of nice idea but didn't quite take off was designed by contract it just really didn't get adopted it was in Eiffel Eiffel didn't sort of take off and you know it wasn't in Java it's not in Scala except this sort of lightweight thing in pre def that people don't tend to use and I think what did win instead was test-driven development so Kent Beck came out and and he's just really good market here you go to lots of conferences lots of exciting talks about this thing like you should write a test first you should ever write a bit of production code until you've had a test right and it's kind of in the same space and I think what happened was that I'll show you a demo what happened was that instead of four and six I actually four and five okay go to four there it is we do do the precondition so that part did happen right but for post condition checks instead of doing it in the code we would do it off to the side and test code so you do those assertions and tests instead of write in the method so the insurance we just didn't do in practice most people don't do that but they actually do something like them over in the test so if you look at this test I'm making a you know I create us I call square root of one and then I say it should equal one well that's actually the post condition check and then I do it for a few other examples right four and nine I know those and so I do that but what that doesn't capture is what the full specification of what designed the contract did it does kind of say it in English so that was one thing that the tests added to is you do have a English or a human language may not be English but expression of what the specification is like the square root function should compute this I mean I should be more I should say within a particular tolerance or whatever it is right but so there's a specification there but what is missing from here is that if you look back at the the if I go back to let's use this one like well no I think right here essentially this requirement and an ensure Clause is supposed to hold for any value of x that's passed in right so it's there's a for all missing and so really sort of take this and move it to a test you need it for all so what that looks like is number go back down to six I think five my five looks like a six but it's a five so instead of this like I just tried three things if you use property based testing you can say for all X's that are of type double whenever X is greater than equal to zero and X is not positive infinity then calling the square root of x should equal mathis squared of X and so what what I did is I got rid of the tolerance I just said actually this one should exactly return that that's my specification should return exactly the same value as long as the preconditions set and then I checked the preconditions farther down and make sure they through a legal arcade argument exception if they fail but that actually is paq because the precondition is the whenever clause and then the action the a is calling square root and then the Q is that should equal thing so that actually moves the whole specification over okay all right let me go back to this guy so and so what kind of one in the marketplace of ideas was instead of this PA Q in the body of the method where you have a require clause and an insuring clause what we do is we say PA true and the method so remember true is the one that's really easy to sort of achieve no matter what you return it's the wild west it will succeed right so that's what we say in in in practice that's what one we do check preconditions we don't take them out which was the default in Eiffel we don't check in if post conditions ever which was also default in Eiffel but we have this thing off to the side where there is a English or human language statement of the specification and then you can do the post can you basically do the post condition checks over there that's what happened so I think in the marketplace of ideas that's what one but you need a for all so I mean basically to get the same kind of level of specification in a test that you got a design by contract you need a for all in the test so I think that's really fundamental part of testing and then the a really in the test becomes part of the cue right so the the P in that one whenever clause is you know as long as X is greater in the zero and X is positive infinity that's the P part but then if you look the the a kind of just gets blended in with the Q I say squared of X should equal X I mean the a is the square root but it's not a separate thing the reason was separate and eyeful is because he had a do part right he had actually implement the body the method we just have to call it right so what that actually looks like is a predicate logic it looks like this thing down here P X implies Q X and what what that is is that it's a it's a thing in math where if X is a is a member of the set of all doubles it has some domain P of X is X is positive and not infinite Q X is the square root of x computes the same value as Scala math square root of x and the way you write that is this in in predicate logic and what what predicate logic has is it's it's like boolean with a few boolean logic with a few extra things just like boolean there's an an that little looks like an a without that they forgot the middle - part that's and also called conjunction and then if you turn it upside down that's or it's called disjunction same thing as two pipes in in boolean right and then the sideways L looks like the name of a ranch that's actually not so that's excavation point in boolean and then you have variables in predicate logic you have variables you can XYZ whatever you wanna call them and they stand for some value in a domain so you know we could think of the domain of real numbers those are like in math that's like infinite but in programming you might think of as doubles and then PQ those are predicates and they take variables one or more and return true or false so that's like a predicate function in Scala it takes what are more variables and returns true or false we call that a predicate and then you can have regular functions which is f G whatever you know F th they take variables one or more and return a value of a codomain so it may be another real number or whatever right that's just like regular functions in Scala and then you have these two other quantifiers ones the universal quantifier that looks like an upside-down a and that sort of means for all and the existential quantifier is a backward Z that means there exists so then you can write formulas in these things so this upside down a X upside down Y P X Y that means for all x and y for all Y P holds for X Y that's what that means in predicate logic and then the second one is for all X there exists a Y such that P holds for X Y right you can make statements like that and then there's two others that two other symbols that are common that can be expressed in terms of the previous ones the right arrow is a conditional it means if then essentially types a if-then in predicate logic it also is called implication and it just means if but the left thing is true if the thing to the left of the arrow is true like f implies G means if F is true then G is true right if then and then you can actually implement that with these preet you know the previous slides symbols by saying not F or G right that actually implements F implies G and then the arrow going both directions called bike additional or its equivalence and that's when F and G have to have the same value so if F is true G is true if F is false G is false they they are equivalent to each other so you can also implement that in terms of the earlier operators and that that means if and only if all right so that's how you say these things in predicate logic and so what what let me back up what basically the this PAQ actually maps to is is that statement of predicate logic px is a predicate it's going to be true or false and if that's true that implies that QX will be true that's the correctness formula and logic his correctness formula okay so one of the other things that had always puzzled me or I thought about over the years with Scala test was what what's the type of a test and what I finally kind of came up with is that it's in its type assertion because it's the type of cue you know the sort of P X implies Q X the last thing you do is Q X that's that result is what the type of a test is and so in Scala test 2.0 we added this this type alias called assertion which is equal to the singleton type for succeeded and so I'm going to show you what that looks like so in if I just I'm gonna know it's I think I actually if I say assert I get what is X X is 33 right so if I say assert X is less than 0 I get test failed exceptions so that's Scala test assert because you get an error message and a test failed exception so you say X is greater than 0 which will which will succeed what you get back out of 2.0 is is succeeded so what's kind of weird about the type of a test is that what you get from predecessor and assume and skull tests prior to to datos assert and and assume there's an assuming there was a unit value it's a singleton object now you get the succeeded singleton object because there's you know if the thing passes you don't really want to know anymore so it's kind of the opposite of production code usually when some the happy thing is you want some information back and it's an error if you don't get it like some and none an option usually it's good news to get a some and bad news to get a none here it's like it's good news to get the none I'm not getting anything but if it fails now I want information I want to know a stack trace on an error message you know what know what happened so it's also I think whenever your assertion fails it represents a bug has been discovered so you don't want to proceed it's not a bug is not something the program can recover from so you actually wanted the program to terminate right there and because otherwise it could do more damage and if it blows up later you might not be able to track back oh where it started so I think you know when you're running a tests it really usually you do usually want it to throw an exception and fail fast because there's no often no point going further in the testers get everything will fail after that you may as well just blow up so what happens is if you always get succeeded it says asserts always gonna return succeeded it's just sometimes it'll blow up with test failed exception so that's that's what the type of a test became on Scala test but what I had always thought might be I mean if the type of a test is assertion then should the type of this-this-this curly-braces it whether you put the body of a test require type assertion and sometimes people have like forgotten to say assert and they just put boolean and that's a bug but it's not caught by the type checker because anything goes basically the type of a test in Scala test has always been any and but it just people do all kinds of stuff and tests they're kind of sloppy they just want to get stuff done so they'll do things like print a debug message or write to a file and then look at the file and that's the last thing in the test or they you know and then there's assertions where like intercept returns the exception if it actually if you intercept you know whatever exception it returns it so if that's the last thing in your test actually that's type exception not type assertion so that would not type checking to be a pain right so I never did change that and I'm not going to because they're just ever sort of like it's actually quite practical to have type any but what I've considered for years and I finally decided to put in was a another batch of styles where the type is assertion so those are going to be called logic styles they'll come in and 3.2 so sometime next year probably oops what was the number yeah here's one hey let's go to nine that's what I want that's was my right here here's an example this is a logic fun spec if you stick the word logic in front of it what happens is now it's the same thing as fun spec except at the type of the test is assertion so if I were to put my print line here you're gonna get this annoying warning for a teeny tiny print - that's annoying it says expression of type unit doesn't conform to the expected types called test assertion and what you'll have to do if you really want to print that is you'll have to say some kind of assertion so we added succeed in 2.0 that's that just returns a succeeded singleton and that's kind of a pain but it would catch if you accidentally forgot to say assert at the end if what you did was instead of a certain X grade in the year you say X greater than 0 then it now becomes a type well there's no X that's also type error but it would be a type error let's say well this work 3 greater than 0 okay see there I forgot assert and it gives me that same type here so anyway that's one thing that's coming in 3.2 okay so the other thing that I like to suggest when people use a require I think it's good to use a require to fail fast and you don't take them out so it's really gives you predictable behavior when when a bug is encountered and you get you know hopefully good stacktrace and error message in the log file but it's a every require is a potential place where things could explode right and so if you want to try to make your software more robust it's good to think about can I get rid of it require by moving that constraint into the type itself right so here I'm taking any old double but I'm saying that double can't be positive and it has to be greater than equal to 0 can I move that into it that type so I started in in galactic adding these thing called any vowels so there's one called posy double it was named at in San Francisco I can say that I named it after Buster Posey because I kind of did because it was hard to come up with like something that was obviously had 0 in it and positive because positive is usually above zero anyway once I say that I can actually remove the X greater than equal to zero from my precondition making that weaker because I made the type stronger so sometimes it's practical do that maybe it isn't but I think it's good every time you write or require it it's like think well can I move this into the type right so let me just show you what how those guys work so what what these guys do I'm going to show you pause and positive integer that's in any bells so if I say pause in 233 I get 1 and this isn't any vowel so it's really just an int 33 so it would hopefully be as efficient as int it's not box unless it needs to be boxed well aunt isn't box unless it needs to be box so you put in sin a list they get boxed if you put pause ends unless they get boxed but if you just write there it didn't get boxed it's just an int right and all the methods on it are really behind the scenes static methods that take an int so so anyway I get 1 if I put a negative number I actually get a compiler error so the this apply method is a macro they can look at literals and say well your valid or not and if you're not valid actually throws an assertion at compile-time which becomes a compiler error that basically it's a compile time assertion so you can't get one with a literal at compile time won't compile but the the downside of that is if I have an X so I have an X equal to 33 if I say pause int X then I get another compiler error because because it's not a literal I can't tell at compile time I can't prove that it's actually positive you know it is so the compiler says use a different factory method called from so from is a an alternate factory method that returns a option so what what happened there was I was trying to help people get rid of requires and I didn't want to just move the problem so I didn't give you a way to get an exception so you can if it's a literal you can get without pain if it's valid but if it's not a literal you have to deal with an option right so you don't you'd have to like say oh if it's if it's about if it's the sum that means I got one if it's a none that I didn't but what I mean I did that in I came here when that came out but since then I've been using it myself and I've kept finding that I needed to call dot get on that option so that that might be worse so what what an example of it is if you look at the the scala dock for scala death that square root it's non-existent so if you look at the source code it says I'm calling java.lang math dot square root so you go look at that and it says I will return a positive number right so I believe I have confidence that when I call you know Scala duck man that square root that it will always return a positive number so why not actually return a Posie double right so so let's just change the result type to posy double the problem is math squared of X is not a literal so I can't have to use that from method so that was an example of like but I would end up doing that I found myself doing that and I was trying to be idealistic I think like you know I don't want you to I don't want to give you a way to replace a require which envelope with an exception with something else that would blow up an exception but I I realized that you know in practice it's just you need one because I was I kept meeting it so then I started think about what should I call it and then I thought well how about unlike the one I picked around most was unsafe from because no one was have a word unsafe in their code so they might it would discourage its use but then it actually hit me that it's an assertion because I but it's a new kind of assertion it's an assertion it's me asserting that I believe it's gonna work see my right here I do believe that's gonna work because I believe in I have faith in the that that Java will always its square root will bring turn a positive number so I think it's always going to work and that's I usually thought that it was right that we put our sessions in tests not in production code and I never used a certain production code I never used a stirring I would put assert in the tests but this kind of assertion I think actually give you something more than other kinds of asserts it gives you a value back you give you the same value back but it's a more constrained type which you did not have before so I think this kind of assertion maybe that's good to put in your production code so what I ended up calling it because ensuring was the kind of thing that returns a result and string if you call insuring on 33 and the assertion passes you get 33 here if you call ensuring valid on 33 point 0 or you pass it in there you'll get 33.0 back but it'll be wrapped in a posy double except that won't be wrapped because it's in any Bell but you get that more constraint type so I thought ensuring work so let me demo that guy so this is not released yet this is it this will be in 3.1 which will come out sooner so that is uh oh I have them right here so if I say pose and ensuring valid X that I think actually worked so let's listen the gate it boom I got an assertion error because it's actually an assertion but I put a nice error message in it so what I think happened is that and this was sort of one of the insights that I think that I came to very recently was that it's actually okay to replace or require which could blow up with a kind of assertion that they could also blow up where you create an N eval like this because what you changed is this require here is a you know search like they require X is positive infinity is that gonna always work I don't know depends of what you pass me essentially that kind of assertion is I have no idea that's gonna work or not so I'm just hopeful so if I get rid of that kind then people calling it I may have like lots of more assertions all over the place but there'll be this kind where it's at a spot where you're sure it's gonna always work and maybe that's better so that was one of the ideas I thought I throw out and and like well throw it out we'll release it and see if people if it actually makes sense to people but it's a new kind of assertion where you get something back that you didn't have before and it would replace requires where you don't know if it's gonna work with an assertion where you're you know it's the extent possible you have faith it's gonna work you have confidence gonna work okay so that's ensuring valid and I already download it so another thing that that Bertrand Meyer mentioned in his book is that it's like PAQ is closest to that of a predicate but he said the assertion language that we shall use in eiffel has only part of the power of the full predicate calculus so I mentioned that the semicolon means and so you have the a without the thing but you actually don't have an or and you don't have a not you don't have a for all you don't have it exists right so you're missing some of that stuff I mean that was also true in Scala test matchers if you take three match or expressions one on each line or you just put them the other in the same line with semicolons that's an implicit and just like an eyeful because it's an assertion of the same thing but there's no way to say or and people have asked for that so I there's nobody or one and there's you can not inside match you could say data value should not be empty that's nodding be empty but you can't take data value should be not be empty and negate that somehow right so one of the things that I'd always thought well I wonder if there should be a kind of assertion that returns a result instead of throwing an exception when it fails because then you could compose them more I mean you could make an aura for assertions but you'd have to catch two exceptions and then there was another one or not you know it's just seems like the wrong tool for the job it always seemed wrong to me even though people had asked for this I always said no but I thought about like well could there be something else that could do that and what I finally decided to release something in 3 2 which is called expectation and I made room for this over the years because expect is a kind of assertion that returns a value whether it succeeds or fails it doesn't ever throw an exception and when install test one that oh I expect was meant something else so I deprecated that and and eventually removed it so that it would be available for this if I decided to put it in but basically assert has type assertion which is always going to be the succeeded value unless it blows up with an exception expect has type expectation which is a type alias for fact this thing called fact and a fact is either yes or no it's really like an observation of what happened so what that might look like let me give you a little quick demo is I have to import expectations I can say I've still got X right so I can say expect X is greater than third well 0 and it says yes 33 was greater than zero so it's like an observation of what happened and the one way to think of it it's like boolean with error messages but I said if true was yes and instead of false it's it's yes comma something some fact instead of false it's no comma something but it always says I mean this the English part always says what it observed right so if I say if my expectation is different if I say X is not greater than equal to 0 I'm expecting the opposite now it's a no because it failed but the observation is the same so that's how that looks and then it's I said it was like boolean with error messages but it's really like predicate logic with error messages because boolean logic is a subset of predicate logic and so what on a factor on type expectation these first five things are just the boolean operators so double ampersand is is the is the and that short-circuits so if the first one is false it doesn't execute this the second one in case there's a side-effect it would not happen there is also a single ampersand on boolean that does both sides no matter what even if the first one is false it does the the right hand side so that's on expectation so in fact double pipe is or and short-circuit single pipe does not short-circuit just like bullying and then the exclamation point just like bullying that's the sideways L that's not but on top of that there's there's two little symbols for implies and is equivalent to it and that's that those two operators so that's if then is implies and if and only if is is equivalent to essentially and then there's there's no for all or exists on expectation but in Scala tests there are several for all's there's one in inspectors like you have a collection of elements one to ten and a list you can say for all X's X should be greater than zero and that will be true or if you say X should be less than zero they will throw an exception right so there's four all there there's a for all with table-driven property checks there's a for all with generator different property checks which is just like generators and those really say for all but we don't test everything we just test some of the values right but allows you to say in your test that you know that's the specification and then there's also a for at least in inspectors where you can say for at least one or for at least three X's X should be greater than three all right you can say that about a list if you say for at least one which is most the times what people say that's the existential quantifier for at least one so I actually made an alias exist which means for at least one and if what's inside of the for all or the exist is an assertion then you get an assertion as the result of for all or exists if what's inside there is an expectation then you get an expectation so these things compose up okay and then there's one other thing that I did want at one point deprecated must because I wanted to one as I thought it was like too close to should and I should didn't one have two ways to do it and the other one was two so I could come back as the kind of a matcher expression that returns a result rather than throwing exception and I remember asking Jonas poner the guy who created akka you know would that be okay because I know you use must and he goes oh I prefer muss but I guess it doesn't matter so that's what he said so I was like okay and I'm I asked around and people like okay whatever you know but boy when I did it I had sort of a mini revolt that's like what must if I'd wanted like you should because it sounds like wishy-washy so it turns out that it did actually mean something different to users so I put it back in and I called it a resurrection so I had deprecated it first and then I resurrected it by removing it deprecated so then I was like well it's cute what I'm going to use for for like the kind of magic expression that returns a result instead of throwing exception and in one of the people who revolted was the other guy in the alka team Victor clang and he suggested will I had never thought of that so it actually fits so now there's will matters which are the kind of matches that return the result they have type expectation and they either return yes or no so so I can show you a couple demos first I can just show you a quick will whoops you can say X is 33 X will be greater than 0 that returns yes if you say well not i'ma have to put friends here will not be greater than 0 then it's that no right so it's it's and these guys you can compose them with those other operators so I could say X whoops I can say that let's go to this guy X will be greater than 0 and X will be less than 100 right that actually succeeds and then they just compose as big as you want so they have and or not the things that matters didn't have right and one of the things that I was trying to solve is that matters were kind of an associate experiment to see if people would like in their code where kind of reads like English in the code and then the error message I just wanted to be English so if you like say X should be well be less than zero then you get this error message that might be what you would say to somebody next to yous like oh 33 was not less than zero and then if you compose them so they say it should be less than zero or be greater than a hundred that's not B be greater than 100 right oh that's the wrong order sorry or it says 33 was not less than zero and 33 was not greater than 100 right so it would compose these English the trouble is it doesn't scale very well so if I do another one like say let's do something that's true B greater than 10 that's true right and B less than a hundred so the Spay put friends here then you get the right error message but 33 was not less than zero and 33 was greater than 10 but 33 was not greater than 100 that English statement sort of you lose where the parentheses were right so it doesn't scale up very well most math 3 expressions are small and it works but you know in the 95% case but then it doesn't scale up very well so what and the same thing is true with assertions there's there's a originally mattres where the way you get nice error messages but once macros came in and we started doing this kind of thing with assertion so I changed this to assert X less than 0 or just make it a boolean expression X is greater than 10 and X is greater than hundred I think that's right I get the same error message so I you can get them out of assertions now but they they have that same kind of scaling problem so there's one other kind of assertions we added the Scala test which was inspired by a language called I mean a test framework called Spock or groovy really really this guy invented this oh it was just really nice now if you you get a ASCII art so there's no more attempt to make English it actually shows the line of code that failed and it just says here's all the values right you can kind of like trace it the problem with this one is that it's it goes horizontal so if you get off the screen and they get all mixed up and if it goes on multiple lines it doesn't work so like if I just actually insert right here let's say that's finished at a hundred we detect that and just switch back to the other one right it doesn't scale so what these guys are if I go back to my expect it it was an attempt to actually do something that scales because you can just these things are like a combination of diagrams and English but the English is only in each node and what what the structure of the diagram the diagram grows this way not this way is predicate logic right I can actually see the structure of the expression there so I don't that's sort of another experiment that I'm gonna see what if people like that or not but it it I think it scales better than the others so anyway that's uh that's these guys and then the last demo I want to show you here is there's the boolean thing so here it actually I just I forgot about implies let me go back to this one essentially because I'm in a logic spec and I use will here I the result of this for all is is actually type expectation it's a fact but because it's last in a lodge expect that requires assertion it implicitly converts it to assertion either blows up an exception or doesn't right but this is the I mean it's kind of readable but there's a logic a way to say that you can use implies remember P implies Q so you can say instead of whenever I could say you know this fact imply this one implies this one P implies Q that's more logically way to say it and you can if you want since they're values you can actually if you like like your P's and your Q's you can just say well P is a fact that is that one that's Q that's a fact than P implies Q if you want to do that and the last one is that I can just actually use double here implies it's not work on boolean I'm sorry I can use boolean in place doesn't work on boolean but it will implicitly convert it to expectation that does have implies and then over here same thing employees takes on other expectations what implicitly converts it so it's it looks a lot like a in in Scala check you use boolean but you have to put these little smiley faces to get a nice error message so it's like that but you don't have to have the smiley faces you get a nice error message by default okay so last thing the conclusion is that this is I think what the essence of testing is is it's a statement this statement of predicate logic which is for all X px implies QX that's what a test is but it isn't about proving propositions which that's what people do use logic for this is more about like doing experiments and seeing if reality meets your expectations so if your expectation is for all X if X is greater than 0 then X plus 1 will be also greater than 0 you actually write that as an assertion that's your expectation what the test framework does is it sort of performs the experiment like a scientific experiment and then gives you a result like this is what the reality is and in this case if X is int then it might say int that max value was greater than 0 but int main value was not greater than 0 because it for hints that doesn't hold for int dot max value because it'll wrap around to a negative number P add 1 so so that's what I think you know essentially it's like a single experiment could prove Einstein's relativity wrong but no number of experiments can prove it true and and you know these little expectations we write are our hypotheses and then these are the experiments that sort of tell if it if if they hold so far essentially ok so that's that's my talk does anybody have a question thank you yes [Music] well you could do that in Scala so one thing let me repeat the question in case it wasn't her it was that there are a lot of programmers who don't aren't native English speakers and what what what about the idea of having the program expressed more in their language rather than English essentially yeah so one of things I did in Scala test because it is about English in a lot of places is I put all those in Internet and in like resource bundles so they could be internationalized but in just eight years we never did it and I can't I mean but it's something that someone could contribute but it's it's one thing I thought it's like that might make sense didn't seem like there's much demand things like if you know those things in languages do tend to be in English and they can be learned but you can also write an if in Scala in a library and have it be a idea Graham if somebody wanted to do that but I'm not sure that's what people want to that level actual error messages in it might be nice to have in their language and I definitely seen Scala tests use where it says you know it you know like describing it but it's in Japanese so people do use their language at least in the you know in parts of their Scala programming yes mr. and InDesign my contract that's very interesting to go about this condition didn't really think about dashes um I you know it's an his book is really big so I don't want to say he didn't but basically let me repeat the question it was did yeah I the question is yeah if it if you can write the test on the code itself does it makes it to have it separate and I think his idea was it's better to put it in the code because it's but it just makes it more obvious what the contract is and I think that's a good idea if I sit back to the Eifel example without freezing my laptop you know it if you look at his like that's what it looks like it's it's it's kind of nice that it's right there because I have to think about it whereas I can just write a method in Scala nothing about it but it just didn't take off it didn't it wasn't what was adopted in the marketplace of ideas the marketplace was ideas for whatever reason they said you know put your assertions off to the side I don't know that's why I say I don't I don't want to say because I his I read his book a long time ago and I just reread the chapter on design by contract in prep of this talk so there may be other things he he wrote about that but he I think if you do this maybe maybe you don't need tests right there right there so I think that was his idea and it just the part that did actually succeed is the required part everybody does that we just don't do this part not just as a festival but in say in production code so if I apply the same question to Scala code and use this as it mean definitely kind of like that now you get louder they're coming to be students no well galactic has got a require but doesn't have an ensure it doesn't but Scala does because I actually it is opinionated I think you should use require but it but think about can I put in a type but the post conditions I think should be in tests I sort of that's my opinion and I think that's what that's what sort of was the mainstream opinion yes any other questions yes yes okay so that's a the question is this we're all work with double and should it how should it work that's a good question so there's there's a couple moves skull sis has three girls ones for a collection for all elements another is a table it like you can make a little pretty table and say for all the rows and then there's just for all doubles that's the one you saw somewhere here like right here for all doubles that one there's an implicit generator pass that generates doubles so that's how that works and Scala check does that and I've been working on a Scala test generator also and so I actually was faced with that question and for example should have present not a number I don't think Scala check does but I actually think it should but was then turning 75 it's it's actually tricky I was doing just some of these tests in a project recently and I kept getting not a number and overflow and it's like I had to like reduce the size of the doubles to use because it was doing some math inside and I didn't know where I was gonna overflow or not and so at the end of it I got it to work I'm like it wasn't very it was just kind of like I got it to work it wasn't I didn't feel right about it I felt wrong felt dirty about it cuz it's like well am I actually I guess I felt is I wonder if I'm excluding actual bugs yeah so yes yes okay so for all does does it does it generate the same sequence of numbers for different test runs it does not but what I what is important is actually you can reproduce it if a test fails so that's not in Scala test yet but we want to add a flag where you can say use this seed so what I think is gonna happen is every test in one run will use the same seed and if it if it prints out maybe I haven't actually implemented that yet but uh that was what I was thinking but I just have a different idea but basically when the one of these tests fails it needs to say this was the seed and we need to be able to say rerun the test using that seed and you get it again but it's turned in by the seed it's a pseudo-random generator that does it if you use the same seed you'll get the same ones but if you don't specify what the seed should be to reproduce a previous failure then it's just gonna say hey give me a seed based on the current time and it's different every time yeah for a random sample of yes that's what it that's what it does well yeah that's what fur all means so in this for all it actually means for random sample but for all is yeah anyway any other questions or comments all right well thank you very much for coming here in the rain and enjoy Scala