What's new since "Programming in Scala"

SBTB 2014, Marconi Lanna: What's new since "Programming in Scala"

Recording: SBTB 2014, Marconi Lanna: What's new since "Programming in Scala"

okay uh can you hear me thank you everyone uh I'm Marone I work for originate uh today I had the opportunity to meet dick wall is Dick here I believe he left so dick is not here anymore but I was telling dick that some eight years ago when I start to learn English I downloaded the entire Java P podcast archive and I was listening to Java PA for like four or five hours a day every single day so if you guys have problem understanding my accent blame it on dick okay uh today I'm going to talk about what is new in scholar what's new since programming Scala uh I have a lot of content to cover here I may need to rush a little bit um so programming Scala is this nice book from by Martin o Lex spoon and Bill Verner and we are very lucky today to have Bill veres here with us uh and I had the pleasure to talk with Bill veres about the Third Edition of the book and he told me that the Third Edition is planned he even talked to me about some of the very nice content content that they're planning to add to the Third Edition but that it's long overdue but the reason why it's long overdue is because he doesn't have time to work on the book which if you think about it it's very fortunate otherwise this talk would not be needed so thank you Bill um we have two additions of the book the green one was published in 2008 and covers Scala 2 7 the gray one was published in 2010 almost four years ago when covers Scala 28 a lot has happened to Scala since then um so just give you a pretty quick Scala timeline uh the first versions of Scala were released in 2003 in 2004 we had version one um in 2006 we had Scala 2 and one very nice interesting thing is that Scala itself was now written in Scala so we had the little dog fooding happening here um in 2007 the lift web framework was released so it seems it shows that seven years ago there were people interested in Scala and doing some nice things with Scala uh in 200 8 we have Scala 2 2008 we have scal 27 which was a very important release then it took two years to release scalar 28 which was a very very big release many people said it should have been called 30 um also in 2010 the play framework version 1.1 uh had support for Scala through a plug-in and AKA was released uh 2011 one year after 28 we had 29 uh and also it was when type safe was created as a company uh 2012 play true was released and this time it Scala was native play was written in in Scala and Java support was kind of a second citizen so things were reversed uh finally in 2013 we had Scala to 10 2014 to 211 and Scala 212 is planned for 20 2016 uh from 2008 to 2016 we had about one scalar releas every 18 months on average which is a pretty good Pace especially when you compare to Java and also what I like is that 28 was a pretty big release 29 was a little smaller 210 was a pretty big release 211 was a little smaller so we are having a tick tock kind of release cycle where you release a pretty big feature have version and then you come with a version that's more focused on improvements and cleanup I really like this schedule uh so what was new in scalet 28 just a very quick overview uh uh it was a pretty big release as I said it brought a redesign collection Library named and default parameters which allowed us to have copy methods for case class which are just awesome uh package objects next to annotation type specialization which is for box of Primitives uh we have Java converter so you could have explicit implicit conversions for collections the implicit conversion would happen on demand you have to call a method as Scala or as Java to make the conversion uh a better rle Scala do to uh lots of improvements to Scala do and binary compatibility for min vers so code compile with 282 would run 280 and 281 and back and forth uh it it's not the full binary compatibility that we all want but it's still something that it's nice to have okay so what is new in scalet to 9 to 10 to 11 uh in a nutshell scalet 29 had one pretty big headline feature which is par collection Scala 211 is focused on improvements on speed improvements for the compiler for for the uh libraries 210 was a pretty pretty big feature release I believe it was even bigger than to rate which is why I have to say that if something is not broken it's SC to1 is going to fix it okay so let's start uh the late in it and the app trade so you should remember the book has a lot of examples where if you wanted to put together something quick uh you just extend the application trait you do not have to define a main method like you do in Java and the problem with this is that the code inside that object is actually part of the Constructor of the object so it's not trade saved it may not be optimized by the jvm and now we have the app trait which is exactly the same you just delete some letters off and the app trate what it will do it will wrap the the the this code inside of kind of a main method so it does get optimized by the jvm and doesn't have the thread safety problems and also as a added bonus you can have access to the command line arguments so let's see if there is really any Improvement here I have a very simple code here I'm just s summing all the numbers from 1 to 4 million uh and I'm extending application and here is exactly the same but I'm extending app so when I run application it takes about 7 Seconds to sum out the numbers when I run app it takes s milliseconds so it's a a factor of a thousand here at first I even thought that the jvm was doing some dead code elimination but I did some more test that's not the case the computation really is happening and we have for this particular case you know we we have lies then lies and benchmarks so this is not even a good Benchmark but it's something we have Improvement of a thousand times here uh let me go back and one thing here if you look we have a for Loop a for thing here and if you remember a few years ago there was this email that leaked from very famous developer that was working for wello startup company uh bought by Microsoft recently and he was saying that Scala is trash that you should never ever use for Loop so I was curious you know what happens if I replace the for loop with a while it's are we going to see an improvements uh and again I'm testing with application and app to see if the jvm plays a hole here uh when I run with for Loop those are the numbers that I got and when I run with a while you can see that for the nonoptimized casee there is a pretty big uh performance gain when you move from forth to while but using the app trait I mean this is not statistically relevant you can assume that it's all the same which brings us to the new feature that is range for each optimization so for now at least for this kind of code where you have 41 2 N it should be just as fast as a while loop we can stop the myth we can stop the fud here four Loops are okay to use in Scala not parallel collections was the big headline feature for 29 uh it is to make easier for you to write parallel code actually so easy that it's not even a joke uh it's efficient and it's transparent you have two methods now added to collections you have a DOT par and a do sack uh dot par will convert a sequential collection to parallel. SEC we will convert a parallel collection ction to sequential uh and all you have to do is called. par on the sequential collection and do your map flat map find collect whatever you want to use high order function you want to use and it's like magic it just works uh depending on the collection dop may be a constant time operation because the data is shared between the sequential and parallel collections uh depending some of them have to be copied not all sack is always constant time operation okay and here are some of the collections that you can use as parallel collections you can use arrays iterables Maps Range sequences sets in 210 you have now trees and vectors those all are parallelized for you on the standard Library so there is some things that you have to be aware of first uh out of order semantics parallel collection of obious obviously will not run the operations in order in sequence so there is no order guaranteed uh if you have side ections side effects this may lead to to raise conditions and both side effects and nonassociative operations can lead to non-determinism um one thing though that it's very easy to get confused and this is important to highlight is that non-commutative operations do work okay so it's only when it's not associative um we had a talk today that told about associative operations but I'm going to give you a quick remind of what is associative and commutative so we have here addition is associative and commutative it works with parallel collections we have something that is associative but is noncommutative like string concatenation that also will work with parallel collections what does not work is something that is not associative in this case subtraction which also happens to be non-commutative it may not be the case you may have no associative commutative operations but if they are no associative this will not work with parallel collections you're going to get bogus results uh so again a very stupid Benchmark here just to illustrate i f a vector with 50 million um numbers and then I try to to count how many of them uh obey to a predicate just trying to do some computation inside the loop not to be too easy for for for the method here and pay attention this is all sequential and boom now it's all parallel and can you see the change I'm going to try to show you can you see can you see can you see just dot par okay four letters and what does those four letters buy your uh for sequential results it took me about 400 milliseconds to run that operation I have a quad core machine the parallel collection took about 100 milliseconds so a quarter of the time on a quad core machine pretty good for four letters I could not do better uh another nice feature that was introduced in 29 it's a generalized Tri catch finally so this allows you to write reusable exception handling code uh so you have something like try and a body and then you have catch and an exception Handler and fine you have a finally and your clean up code uh usually this has to be a block with code inside but not anymore you can block and clean up can be anything and Handler is a partial function so what means that you can write a method or V or whatever there is a partial fun function and then you call try something catch and it just give the name of that partial function you do not need to write the all your catches again again again over again you can reuse very nice but unfortunately it's pre useless because uh we saw today that functional code should not throw exceptions we should not be catching exceptions anymore we should be using tries the Trion ad to handle errors for now one so let's try try this monad uh that encapsulate an exception it's an abstraction over an exception um so you can use it everywhere an exception may occur in your code uh try may be either a success or a failure so it kind of looks like an option or either uh where either you have left and right and option you have of course some and none but none doesn't have a value in this case it's closer to either where I may have a right and a left value in this case it's called success and failure and it only gets nonfer exceptions system through and it's a monad so I have map and Flat map to to to compose tries I also have recover and recover we which are map and Flat map for errors so use map and Flat map for Success recover and recover withd for failure uh we have filter get lse to option all very useful methods here so let me give an example I have a method that takes two strings it will try to pass the string as an integer and then we'll divide one by the other uh so of course the result of the the for Loop is a try itself so what happens when I call D A Zer I get a number format exception a when I call 1 B I get a number format exception for B when I call one Zer I'm going to get a division by zero uh so you see this is all failures I I got back an object it's not an exception that blow up uh it's a value that I can forward or do whatever I want and of course we have here I can if I don't care about the errow I can convert it to an option or if I have a reasonable in this case it's not reasonable but if I have a reasonable default value I can use get else and there is the method get which I think it's very useful uh get has a bad name because of options you should not use get on options but for try get can actually be very useful let's say you have a code base and you wanted to migrate your code base to use try now instead of throwing exceptions so on the first step you write your code using try and at the end of your method you put a try. get then your method behav just like it did before you know it will throw an exception you return a value and when you have the time to change your API to be returning a try instead of returning the plane value you just get get rid of the g at the end and the method it's converted to the new API so get I think it's very useful when you're are transitioning from exceptions to try and you cannot do it wholesale you can have to move little by little uh value class in place class and extension methods in place class uh it's just a more convenient syntax to Define um extension methods uh implic conversions okay they have a Constructor with one parameter so this is an example of an implicit class I have class A that takes a integer and I may Define as many methods as I want this is transformed by the compiler in into a class and an implicit method so that guy will become Class A and an implicit method just like it was before so it's just syntax sugar to make it a little simpler for you to create your own impl conversions okay uh and because the method has the same name as the class you cannot case class cannot be in place it's a limitation that we have right now a very simple example here I create a implicit class in Ops and I Define a method is Stars and if I call five do stars I get back five stars as a string um value classes uh they are used to avoid object location conditions apply it's not always in some case the location does happen uh it they give you the type safety of kstone data types without the runtime overhead uh this is very useful for things that usually would be a primitive like a double an integer a string but that they should not mix it's a case of apple and orange so temperatures in C and Fahrenheit you can create a value class for Celsius uh things like email and first name there are strings but you don't want to assign an email as a first name so you can create value classes for that uh they only have a primary Constructor with a parameter uh they can only have methods they cannot have any other type of members uh they may not overwrite equals and hash code and they cannot be extended by other classes uh so here's an example that I have a case class age extends and invol so this is the Syntax for you to declare a value class you extend and Evol and I create an object uh of type age and at compile time age is going to be of type age but at real time it's just an integer a plain old integer so if I try to do something H + one I'm going to get a type mismatch even though at runtime age is still an integer now the really nice thing happens when you combine imp plus class and value class uh then you have extension methods um which provide you allocation free okay uh so extension method is equivalent to use an object with stat helper methods uh it's just a mechanical transformation performed by the compiler so an example if I had an impl class just a regular impl class like on the first example and I call Five Stars this will be equivalent to creating an instance of that class and then calling the method stars on the new object that was created and then the object is immediately throw away uh but if I use a implicit value class what what happens when I call something like five stars behind the scenes the compiler is going to turn that class into an object with the method and you see that the method signature has changed before it would take no parameters now it does take an integer as a parameter and when I call Five Stars what I'm actually doing is calling the star static method on the in OBS object and passing the integer five as a parameter can you guys see here is it okay okay uh string interpolation so string interpolation really it's something that most if not all script language provide but in Scala they're a lot lot more powerful so you have seen it already I can just put a dollar sign in a variable name and then it gets interpolated in this case there is really no difference using a string interpolation from using string concatenation just a different syntax but Escala does have a few tricks that other language don't offer first uh you can put anything there it doesn't need to be just a variable so I could do something like uh 2 + 2 and the result gets computed and this is what is going to be interpolated some language do that uh it also works with triple codes and it's a shame here that the the the the variables are not highlighted but okay I I hope that I can make a few million dollars with this code feel free to copy um but again all of this is we have seen in other language okay um what we do not see any other language are H okay so sorry uh if you're curious how do you escape a dollar sign you use a double dollar so just my two sense here um things that other languages do not not offer so first it's formatted strings so I can use something like f print F from C or Java I can put a format string and now my double there will have just two decimal places because of the formatting that I have specified no big deal right well actually big deal because the f interpolator is typ safe if I try to do something dumb like format the double as an integer I'm going to get a type mismatch at compile time not when I run the code but when I try to compile the compiler going to say to me that I'm done so this is pretty awesome I do not see any other language that does that um and also you can Define your own interpolators so we saw the S interpolator we saw the F interpolators uh we have not in Scala but some libraries like slick and aor from play framework they Define a SQL interpolator and how is this different from just a string interpolation well the C interpolator will do some uh C escape on the variables so you do not get a c injection attack uh we also have Json interpolators and we have dozens and dozens of libraries that have defin A J interpolator how is this different from just string concatenation well they will do the Json serialization on all the variables so if f is an array it will transform food to ajason array if bar is an object it may serialize bar to ajason before it interpolates it's we will not just call Bar to string we will actually try to transform bar into a Json object and put it there so very very powerful feature to be customized uh okay uh Futures and promise uh so future is a way to perform men's operations in parallel in an efficient and nonblock way uh features a place holder for a result that does not exist yet but may become available at some point and the way you're going to use this result is you attach some uh some callbacks to your future uh you have uncomplete un success un failure and those callbacks are going to be executed eventually okay uh the order in which they are executed is not deterministic and they may not not even be executed in sequence you may have then executing in parallel if you have more than one callback on a feature when that feature completes probably many of them will be fired in parallel uh features are monad so they can be combined and transformed with all the functions that we are used to and they can be used in for comprehensions okay so I'm going to give you an very quick example here I defined a pretty simple feature it prints when it gets started then it's slips for a little bit and then it prints when it ended okay and just to return something it returns itself its name no big deal and I have seen people doing this a lot when they are trying to to to use a lot of Futures and at first there is nothing wrong with this kind of code where you put a lot of features in a for comprehension the only problem here is that if B does does not depend on any result that is computed by a this is going to be changed so one future will not start before the previous one finished so when I run this code with that future I get something like this a started a ended B started B ended so it goes sequentially what I do recommend for you if there are no dependencies in the Futures is for you to create the features first and then you compose them to get the final result and when I run this code this is what happened all the features got started and can see that in this particular run they did not even get started in the order they were fired this may happen if you run again this is going to be different uh and they end in this case they ended from the the shortest to to to the longest one I I don't know if this is going to always be the case probably not um so promises uh future is a r only place holder a promise this is a writable single assignment containers that will complete a future uh just show you an example here you create a a promise and then you get a future from that promise okay uh and we can see that the future is not completed yet the promise was just created then on real code this would be probably executed in parallel inside the future maybe when you have the promise is done when you have the value for the promise you call promise success and you pass the value then that future that you stract at the beginning will be completed so this is one way for you to create your own Futures to fulfill features that you create one last thing that I want to say about features um when you work with features it's very common that you're going to have a list of features a sequence of features uh the future. sequence method is very useful because it gives you a future sequence which usually it's a lot more useful for you than to have a sequence of future so just a little tip here that this is very useful in practice when you're working with features uh the dynamic trait and there is a reason why I'm saying tra I'm not saying Dynamic type okay because this is not Dynamic type this is just syntax sugar it's a simple mechanical transformation performed by the compiler did I see that already today I I believe I did uh Scala does not have any sort of dynamic type Scala doesn't have any sort of optional sta type there is no such thing in Scala this is just syntax sugar um it enables flexible dsls if you want to create your own dsls and also may be convenient when you're interfacing with Dynamic language like JavaScript uh and you're using data formats like Json uh so what do you do you understand the dynamic trait and you implement at least one of the following methods apply Dynamic I apply Dynamic named select Dynamic update Dynamic and then what happens you can call something like um a property access so food. bar this will be converted by the compiler to food. select Dynamic bar so your method select Dynamic is not responsible for handling this request uh you also have assignment you can also have an array like uh Syntax for assignment you can call methods and pass as many parameters as you want you can have support for name it parameters this is pretty cool um and of course you can combine this in any way you want so is it useful I don't know it may be uh but please some people like to to bash the dynamic trait saying that this is dynamic type it is not keep that in mind uh AA actors uh so in 20 210 AA is Now the default acto Library uh in 211 the old um acto library was deprecated too many details I don't have the time here uh go to the book read the guide if you want to know more detail sorry about that guys modularization um so some of the more advanced features in Scala now they have to be enabled you cannot just use them uh and how do you enable that you call import language. X where X is one of Dynamics extensional higher kinds imp Place conversion post F Ops reflective calls or experimental

macros okay so if you wanted to use any of those features you have to put an import to enable that fature uh you can also pass a common line option to the compiler um and to enable all features you just put put an underscore and all the features are going to be enabled okay and one thing that I want to to call attention here is that the Imp Place conversion is for you to Define you implic conversions it's not for you to use imp implic conversions that are already defined um not need to use and I don't know but if you're defining Place class you don't do not need to import so I I don't know really what the reason for that implicit method needs to be enabl implicit class does not but that is what it is okay reflection macros and qu code okay so now finally we can have runtime errors at compile time um so this is for meta programming uh programs that modify themselves at compile time again for code generation Advanced dsls uh you have runtime uh reflection and you have compile time reflection through macros okay and the problem is that the Java API does not support all the features of Scala so Scala had to create its own reflection API that expose the Scala um elements that Java does not support uh quas codes were introducing 21 and they are significant improved simplified notation to manipulate Scala syntax 3es and this is really awesome so I'm going to give an example a quas quote is just a string interpolator sorry so we have something like this uh f plus bar so how is the Q interpolator different from just the S interpolator just a a standard string interpolation the difference is that I'm going to parse that code AS Scala code and build a syn sytax three okay uh in this case I don't do not have actually interpolation happening but I could have Dollar Bar and then the content of the bar would be put there and so when I have a quas code it has the same structure as a piece of code that would produce the same syntax tree so this is true f plus bar has the same structures of food do bar parenthesis method of call even more powerful yet quad codes can be decomposed using pattern matching so I can have something like um dollar f plus dollar bar now I'm using some interpolation and if I do a a pattern match on 1 + 2 * 3 four will have the same structure as one as the literal one while bar we we have the same structure as 2 * 3 and doesn't matter how I write 2 * 3 is still the same syntax threee so this for you to write your Macros is awesomer than awesome it's really awesome um I'm going to give an example of how to write a macro very quickly um so let's say that I want to have in my code log we have a problem and I want to write a macro that will transform that to if logger is enabled then you call logger.log we have a problem okay um so what I do I Define a log method and it's just a stub I I say that this method is going to be a macro this is the actual macro itself and as you can see with quasi codes it's trivial it it's just like writing an HML template I just write the Scala code that I want to be generated I do an interpolation that is needed and now I have my macro okay some examples of how macros are used in the world for Jason um I'm not going to spend too much time here but everything happens at compile time there is no runtime reflection uh in in play you can define a case class and then you you there's a macro that will create a serializer and a deserializer for your case class to and from Chason uh Scala pickling is a new library for for serialization it's about the same um has even less boiler plate than the Json API okay uh case classes with more than 22 parameters really I have nothing to say here if you need them you need them if you don't avoid them but go crazy you know your day is now it's your day if you want to be my guest uh new methods and collections I I I didn't have time to to research a more extensive list sorry but so this is far far from being exhaustive here are a few ones that were added uh I personally I love love from the bottom of my heart option contains you have no idea how useful option contains can be uh sa permutation combinations this has made my life so much easier at job interviews when people ask me to comp put all permutations off a list T done I got the job okay uh so yeah uh SBT incremental culation says the legend you know I once heard a old wise man tell a legend that if oh okay yeah yeah yeah sorry sorry sorry there's a joke so Scala 21 compiles 10 times faster okay but your build time is still going to be the same the compiler going to use the time to to mind Bitcoins uh so says the legend that if you use BT 0132 or later and if you put this magic incantation in your build SBT you put this line they said that when you have to recompile your code base this is going to be faster I do not have any real data data here to share with you guys if anyone has I would love to hear about it so please speak up after the presentation um last but not least pr. triple code uh it's just a placeholder for methods that were not implemented it's very useful for tdd thanks for reminding me Bill uh code samples presentations blogs or if you have a a trade through of abstract methods that you need to to extend and you don't want to to to to implement all the possible methods so you just fill the whole page with triple codes and you're going to do your job so this is what it looks like this looks like code that doesn't compile it looks like sudo code but it does compile it does run of course if you try to call a X you're going to get an exception this is not something that is going to convert the Rubik croud to SCA but it's still may be useful okay so what we have in the future f what I'm going to be talking about when I'm complaining that the fourth edition of programming scal was not released yet uh we're going to have in2 we're going to have Java 8 support so we're going to have Java style lambdas or sidas or Sam or whatever they want to call it Java streams um and what is cool is that's going to be B directional inter interoperability Java may call um sorry uh closures that you define in Scala and vice versa so you can have all of that um but unfortunately it means that Scala 212 is going to be Java 8 only okay uh they are promising a compiler based code style Checker which may be good for enforcing best practices uh improved l v initialization so for those of you who do not know L vs are the sausage of Scala okay they look delicious at first until you learn how they are made so they are promised that they yeah yeah yeah so Las vs they have a performance penalty how am on time okay okay so they do have a performance penalty not only when they are initialized but every time they are accessed after initialization so they are promised use some Java Eight Features to try to improve matters here uh spores it's closures for current and distributed environments Ayn and a waight async and a weight is something to simplify Futures future composition it's uh yeah just that simplify Futures they're promising a library a collection Library clean up and simplification uh we will not have the syntax procedure anymore you have to put your equals all the time you cannot omit the equals there uh who here is a lift user don't be shy raise your hand oh come on guys two two guys okay sorry for you guys but uh XML literals they they will be off the language now you're going to have to use a string interpolation for XML uh Scala meta which is things to make micros yet more powerful and easy to use uh Scala JS is a JavaScript backhand for Scala you may also have want to look at d uh it's experimental project where they're testing new features for Scala okay uh I don't know if you can read but the code and the slides for this presentation is at github.com Maron L it's my first name my last name if you don't remember you check the the the conference page there just put github.com myy name this will be published uh at our blog at originate okay I have all the reference here for you uh there links I cannot click the wall but if you want to know more about some of the features I talked about here other the reference so thank you very much for your patience and do we have any questions