Scala: The Simple Parts

SF Scala: Martin Odersky, Scala -- the Simple Parts

Recording: SF Scala: Martin Odersky, Scala -- the Simple Parts

so thanks for for having me here and it's great to be back at workday I think workday was our first Contact in the Bay Area when we started out with Scala and tried to look reach out to Industry so it's really good to be back here now um before we get started I just wanted to do a quick poll here who here is actually a Scala developer no Scola okay so basically everyone right good great so my talk will be oriented to scalar developers so that we we have the right match here in fact we Scala is now 10 years old we have an anniversary that year um it came out the first announcement was on 20th January 204 and it's interesting to look back and say well when we came out with Scala what did we think uh What uh what what did we want to do of course we would never have dreamed to have a language as big as that uh but it's true that at the time what I wanted to do is figure out whether we could combine functional and object in programming and make it practical that was my first uh research agenda at the time just to figure out whether it was possible and then in the announcement says well that's essentially the points that we actually wanted to highlight the first one said abstract types and mix in composition they unify ideas of object and functional programming so it plays in the modular modular area here pattern matching over class hierarchies unifies function and oop and it greatly simplifies the processing of XML trees so XML trees were very important back then that's probably something that we would revise nowadays uh but at the time XML was really extremely hot everybody wanted to do something with XML and for me uh the attraction of XML at the time was was you know 2004 we were still in the time where object oriented was the industry there was nothing except object oriented that was it the dogma and the problem there is to say functional is different because functional says well you we have our data here and our methods or functions over the data somewhere else and we access it with pattern matching and the object oriented Dogma said no you should not do that this is completely wrong you have to put the methods where the data is you have to put them right in the data and XML Al sort of the poster child of something where you say you can't do that that here at least for XML this approach is not possible because you can't put methods in XML trees XML is pure data so XML was sort of the thing ah finally we can use pattern matching and functional methods for this sort of thing and since XML would be important we put it right in the language and then number three was a flexible syntax and type system to enable the construction of advanced libraries and new domain specific languages and certainly a lot has happened there in domain specific languages so it's interesting to see where we are and where what what the initial announcement was now 10 years later are we grown up well Scala has definitely grown a lot I think by conservative estimates we have probably around 100,000 developers now worldwide uh our coera courses on uh principles of functional programming and reactive programming together got I think so far over 200,000 subscribers and if you aggregate the Redmont rankings that's get GitHub projects and stack Overflow questions and Scala is at number 13 which is actually quite respectable but more than these numbers I think the thing that matters most is that we have many many many successful rollouts and happy users and that's fantastic but Scala is actually also discussed probably more controversially than usual for a language in its stage of adoption and the question is why why why why is that why is Scala so controversial even though a lot of people are really happy with Scala and uh and and and have and are very successful with it so I think part of the problem or part of the situation is that we have quite a few internal controversies different communities of the language don't agree what programming in scolar should be like and then there are the external complaints which go all over the place really so that's that's one you hear a lot Scala is too academic uh that one you hear also so Scola has sold out to Industry uh scola's type types are too hard it's pretty common but some people think that scola's types are actually not strict enough and I guess a lot you hear Scala is everything and the kitchen sink it's just so so big and so huge and that's actually the comment that vexes me most because when I designed Scala I wanted to do basically exactly the opposite I wanted to do something that was very simple and very regular and uh I've been fighting a lot uh both in the message to say well how how is it simple or complicated and I've been fighting essentially in practice to keep it simple so this talk is part of my mission here to say well I want to really figure out what are the simple parts of Scala so this whole thing I believe is our signs that we have not made clear enough what the essence of programming in Scala is and that's what I will try to do in this talk so picture so far uh that's one where that I shown in almost all the talks I've ever given so I said Scala is a unifier it unifies object oriented and functional programming and that gives a language that is at the same time quite agile you can use it as a scripting language in fact we have even won the script Bowl one or two times at uh Java 1 and uh at the same time it's very safe and performant it's sort of the backbone of a lot of services or the whole company companies like workday or Twitter or guilt or many many others bet on Scala not just for the odd application but for the infrastructure good so Scala then uh as the name implies is a scalable language it does this to be scalable from scripting to massive systems language scalable you could ask what is that well the direct meaning of scalable would mean it's a growable language it's a language that can be molded into new Languages by adding libraries and uh these libraries could give new languages that are domain specific or general languages and if you want to find out more what growable means and I the best uh uh talk to go to is really this talk by Guy steel oopsa 1998 growing a language it's a absolute piece of beauty if you haven't seen that talk yet then I can only warmly recommend that you look it up the second meaning is uh of scalable is enabling growth so not growable as the language is itself but the language would enable growable software can be used for small as well as large systems and it allows for a smooth transition from small systems to large systems and that's very important because I guess in many situations large systems have started out small and uh there was no clear break in between so Scala gives you essentially one feature set that works for small as well as large and that's important for that so let me go to the first meaning first A growable Lang language so Scala definitely is a growable language it has a flexible syntax flexible types user definable operators High functions implicits and all these things together make it relatively easy to build new domain specific languages on top of Scala in fact I believe the original argument for workday was precisely that that we could do that uh that we could do a domain specific language for workday application and nowadays where that fails you can always turn to the the macrosystem where you have even more freedom than with all these things even though so far the macrosystem is labeled experimental but so far that hasn't H many people back and uh what you the the it's not only true in theory but that really has panned out there's now a large number of domain specific languages or Advanced libraries that run on top of scal I give you just a a small sample so there's chisel for Hardware that's developed over at Berkeley there spark of course for Big Data uh the spray for HTTP dis dispatch there the shapeless and Scala Z libraries for type level programming there the database connections slick on squel and several others there the testing Frameworks like specs and scalar test there's AKA for actors there's SBT for the build tool all these uh tools or libraries there are really domain specific languages on top of Scala so you could say this growth strategy has really worked well but is growable always good in fact you could argue it's a double-edged word because uh the domain specific languages actually have often the effect that they fracture the user Community there's a famous essay the Lis curse about another language that is very growable and that has suffered from this problem to some degree problem is it's if it's too easy to make new languages on top of a host language then everybody will do it and you end up with a lot of different dialects and those dialects will have problems talking to each other or the people using the different dialects will have problems understanding each other besides no languag is like by everyone no matter whether it's a DSL or a general purpose language and that means that even if you do a DSL and you find it fantastic on top of Scala not all Scala users might agree with you never mind all the other users out there and finally host languages do get some of the blame for the dsls they embed so all this is sort of a double s growable is good but um there are problems with it it's great for experimentation definitely but for production use it Demands a lot of discipline to achieve Conformity it demands agreement between the partners in a team or between different teams in Conformity so that was the global aspect the second aspect was a language for growth and I think that's all true also true for Scala because one can start with a oneliner in the repple or in the worksheet one can experiment quickly and one can grow with all without falling off the cliff the language stays with you the same constructs stay with you and of course that has worked out also well from very small Beginnings scalar deployments now go into the millions of lines of code uh the language where works for very large problem programs it's true that the tools are challenged the larger the code base the more pressure you have on the build times and that's definitely one of the priorities where we work very hard to improve that in the 211 version of Scala that just got released we have drastically improved incremental build times and hopefully down the road we will also have some progress to report on the on the uh clean build times a large system and definitely we have large systems in Scala now by the way a good quote that I heard for that is a large system is one where you do not know they do not even know that some of its components exist so what enables this sort of growth in Scala well I believe it is the combination of object oriented and functional and if I look at what I can see of the large systems out there then they definitely do use both aspects they are predominantly functional uh I think every all most of the successful large roll outs of Scala are predominantly functional but they also make good use and extensive use of the object oriented capabilities to structure the large systems into objects classes traits and so on unfortunately there's no established term for this combination of object oriented and functional some people have proposed object functional but I don't like these hybrid names they uh remind me too much of pl1 besides uh we are in a difficult spot here because while we would prefer things to be like this watch the set in Perfect Harmony uh unfortunately it's not often like this it's more often like this but that's how functional people often see oop and that's how objectoriented people often see functional the mad scientist who's at best crazy and at worse dangerous and unfortunately Scala Community often Finds Its self between the two chairs in the middle universally loathed by both sides so that's where we are um another quote here which which plays uh in the same direction is uh James iy from a brief incomplete and mostly wrong history of programming languages if you haven't read it yet I can recommend it it's hilarious so what he wrote here is 2003 a drunken Martin neski sees a re peanut butter cup at featuring somebody's peanut butter getting on somebody else's chocolate and has an idea he Crees SC language that unifies constructs from both objectoriented and functional languages this pisses off both groups and each promptly declares Jihad amazingly prean no I I wouldn't would have thought he I think he wrote that in 2011 but the definitely this Jihad bit has has evolved quite well since then so yeah so object functional it's a difficult spot because the two communities they don't really understand each other sometimes they even say there can't be a compromise you can't merge the two even though I believe we have successfully demonstrated that you can and what you need to do all the time is overcome this misunderstanding and that's sort of an unproductive thing so what I'd like to do here is to say well let's shift the Viewpoint a little bit and say well what is it this combination of object and functional that is special about it and I would like to propose that the word modular for that so Scala is a modular language it unifies object oriented and functional to be a language where you can go from small scripts to large systems and the idea is modularity so what is modular programming well if you look it up then modular programming you compose your programs from modules and what are modules well the best definition that I've seen is a module should be a simple part that can be combined in many flexible ways to give interesting results simple parts that can be combined in many ways think Lego bricks that's one one good example of a module out there and simple usually means that it encapsulates only one functionality but you could say well that's old hat should we go back to modular 2 that was a language that uh uh use this term modular programming I see two thumbs up yeah yeah well the four thumbs up I'm I'm I'm one of the the crowds who actually started with modular 2 to do serious programming and I I really very much like the language when it was there but that was the early ' 80s so or mid 80s so should we all go back well of course not uh the early 80s were quite different from now in particular programs were were much smaller and they were more constrained to the so-called fman Paradigm that means your program consisted essentially of variables that modeled memory cells and then there was a process and there was a bus of a single width and that the programming languages were like that they were speaking to Dore variable by variable in pointer by pointer you did this piecewise bit of modification now if you want to scale that and I say you you say you don't want to talk about pointers or ins you want to talk about things like graphs and customers and polygons and pols and things like that and if you do that you need a theory for these things and if you look at theories of polinomial or polygons or graphs then they are functional that functional says well for instance if I have two strings and I add them I will of course get a new string which is the concatenation of the two strings I won't like mathematics doesn't tell me Well if I I can change the third character of the string and it's still the same string no no it's a different string right it's obvious you could say but of course the old programming languages were not like that that argument that I've just made was made by John bakos who's uh the inventor of arguably the first high level for Norman language that's called fortron and when John beos got the touring award the highest award in computer science he gave gave a speech where he said well essentially fortron is no longer the right way to go about things it's functional programming that was in a sense the start of modern functional programming in the late 70s and the other paper that is there very interesting is from John use why functional programming matters I don't know who has read that paper at some point quite a few so if you have read that paper then you will be with me to say well that's essentially a modularity argument in that paper John used takes an algorithm which is essentially a loop where you mangle the the the loop and the termination condition and the iteration step and the rounding and he essentially splits apart the individual things into functions by themselves and and recomposes them and says well that's much more modular because what was essentially all mangled together meshed together in one loop I now have a have a separate functions okay so functional is definitely good for modular programming uh but it's not synomous so it functional does not imply modular necessarily there's some Pro Concepts in functional languages that can be at odds with modularity so for instance anything that assumes a global namespace like haskill type classes wouldn't be very modular in in a hasale type class you can Implement a given class only once with a given instance type for the whole program now if you I just said a large system is one where you don't even know that certain components exist so that would be bad news for type class implementations because in some components that you don't even know exists you might have an instance definition that conflicts with yours and then your program wouldn't compile anymore and or wouldn't run anymore and there are several other things as well that you can argue about objectoriented languages are in a sense the successor of the classical modular languages but object oriented does not fully imply modular either so there things for instance like monkey patching in Ruby which also suffer from this globality problem so you can essentially add a method to the root class object but you can do it only of course only once if somebody else added a method with the same name you get a conflict and you have problems not exactly modular besides uh object orend programming often relies too much on mutable State and that gives hidden dependencies between modules that uh uh makes composition difficult and a fairly weak facility for composition and decomposition compos V composition facilities means that you typically have to rely on external dependency injection Frameworks to get anything done and that for me is a weakness of the language itself if you require something else to do something as fundamental as dependency injection then obviously the languages themselves haven't done done enough for you uh to to to satisfy you here okay so Scala then before I discuss the part the parts that are more have more to do about modules I just want to start with the simple Parts in the language itself so what are the simple parts that can be combined freely and in fact I've thought a bit and I think they these parts correspond to the fundamental actions that programmers when they write programs in Scala do every day and those actions are the ones that you see here and we will go go through them bit by bit so each one of these actions corresponds to one language constructs that supports it and as always just cave that simple is not easy that means for some people uh these things might look easy that means familiar for other peoples it might look strange but that doesn't say whether a thing is simple or not simple means one functionality can be combined well with other easy means it's familiar it's close to what you already know and the two of course are not necessarily the same okay so action number one what we do is compose we compose a program from parts and that is made in Scala very easy because everything is an expression and expressions are composable uh you can put an expression as the oper of an operation as the argument as a function as the condition of an if as the body of a try and so on and that's of course helped by the fact that all the things that would be statements in one of the more mainstream languages are expressions in SCA so the if else is an expression it produces a result so is the match the analog of the switch in C or Java so is the try and that means that you can put everything inside everything else at complete freedom and you don't need to drop out and essentially have statements that you write one after the other and that are then connected with mutable variables so that's a number one thing which is often overlooked it's completely trivial but I think the fact that you have that is very very liberating in the way you can write your programs you can refactor your programs because it just means you have this complete general concepts of comp composability okay so the Dual of compose is decompose or I call that match so when you compose data some often you also want to take them apart you want to find out well what do you have what have you composed what's the structure of this thing and act accordingly and that's in Scala done through pattern matching again a functional feature that typically functional languages have um and that's a dual of composition so here's a classical example of pattern matching I guess you've seen many like that uh we have a the structure of arithmetic Expressions so there would be a trait expression and then there would be two case classes one for numbers and one for plus operations that both are expressions and then somewhere else we can write a function like eval which takes an expression tree and gives the uh integer that it evaluates to and that of course would be a match which says well if it's a number with uh the value n then that's the value of the expression if it's a plus with the oper island island R then evaluate both operant and sum them completely trivial so that's a very simple and flexible way to go about things uh uh pure functional programming people coming from hcll or cam say would sometimes criticize that this is a bit verbose and that's true compared to uh the algebraic data types that you find in these languages this thing is a is a little bit longer on the other hand to to say in in scalar side of the argument uh we don't need a separate type so these are just class hierarchies like other classes as well so that's more uniform and it's also more flexible it turns out that with the subtleties of type parameters or or other things in hcll or camel you sometimes have to use a different syntax to achieve something which is all uh the same here and by the way and besides uh the uh how how big is the part uh how big a part of your program would be played by these data type definitions normally probably is a tiny percentage 1% maybe 5% Max or something like that so whether you spend a little bit verbosity here or not it's for for me it doesn't really matter in real systems good so that was sort of the functional approach to these things I think it's worthwhile to actually look at the traditional object oriented alternative that you can also do in scalar so that would be the traditional object oriented alternative you have the trait expression and now the eval method would be right in the trait but it would be abstract here and there would be implementations eval equals n in the case in the number case and that summation in the plus case so that's two ways to express the same thing and that way to go about is okay if there are only a few operations and the operations are fixed and it's even the recommended way to go about things if there are a variable number of cases so if what your future growth of the system would consist to add add actually new expression variance and the number of methods that you have an expression stays fixed then that's the right way to go about things because to add a new variant all you have to do is add a new subass and you don't have to touch everything else whereas previously to add a new variant you'd have to touch your cases here add a new case to every switch on the other hand in a lot of cases you don't have this situation you have the situation where the data is relatively stable and fixed but what what you want to do with it might vary widely so think about the data being an interchange format or a a uh a database schema or something like that uh these things are normally given and they last a long time but what what you want to do with the data might change every day so in that case I would say that the functional approach to say keep the business logic and the data separate is definitely the better approach Scala is some people sometimes think that Scala is confusing because it offers both approaches and uh my only answer there is well there are problems that demand both Solutions and uh I sort of appeal to you as the engineers to actually make the trade-offs and pick the right one uh but uh of course it would be simpler to just shrink wrap one and just force it down you and declare that this is the best way to go about things always which unfortunately isn't true okay so that was number two match number three I would say is grouping so the idea is that everything can be grouped and nested inside each other and there a static scoping discipline and that again is uniform it's the same thing for everything whether it's a term or a type both have exactly the same scope rules and you can Nest everything inside every everything so here of course you have a little program where you have a a predicate isolution which is a method and then you filter over that and the isolution method is right in the solutions method in Java of course you couldn't do that you can't Nest a method inside a method on the other hand you can Nest a class inside a method and then a method inside the inner class so you can Nest methods inside methods if there's a class in the middle but not directly why don't ask me why good so the idea here is that this grouping principle is actually very important because it prevents or it helps people from avoiding a rookie mistake that one often does when one is new to functional progr programming so the rookie mistake is just to pack too much in an expression to say well because I can compose so well I can put all these things in my expression uh I will do it and I will end up with Expressions that are positively huge so that's the thing that I actually found in our own code base so we definitely are as guilty as everyone of this thing so it's a very impressive piece of code it achieves a lot quite amazing what you can achieve in a single expression but it doesn't mean you have to do it so my tip here is to say probably if people want advice how can I improve my code base how can I sort of do the right way to go about Scala here's the tip that I think has the most impact find meaningful names so make sure that every intermediate result of your program has a meaningful name if there's too much uh or if you can't find meaningful names for these things then it's probably a code smell it might mean that your abstractions are actually wrong if you can't say what it is and communicate what it is then it's probably wrong so what I did with this simple with this expression that you've seen I tried to find meaningful names and I believe the refactoring it got slightly longer but it it also got clearer so we're talking here about some sources uh it's in a compiler so these are compiler sources there's a workpace route there's a method that gives you for an entry the sets of files that correspond to it and then the find the expression as a whole it takes the sources and it for each Source it uh determines the files that corresponds to the entry and it concatenates them all so okay so now we're in business now we understand what this thing is and of course I can put that in brais and make this an expression uh I don't have to I don't have to trade off the fact that I have these intermediate results which are now well named with the fact that I have to put them maybe in the enclosing class and have confusing things in the enclosing class scope I can pick the natural scope where these things are and that I think makes the advice more powerful because there's now no downside to create things that have new names so number three that was uh Nest group number four would be recurse so in functional programming of course uh recursion is everywhere um recursion lets us compose to arbitrary depths that's why it's important without recursion you you you the depth in which you could compose expression is always finite and recursion is I believe in a functional program almost always better than a loop in particular because you can make it just as fast as a loop that's the other nice thing because uh there is a thing called tail recursion and if a function is tail recursive then it is uh for all intents of purposes implemented just as well as the fastest glop you you you could write and you can verify that a function is TA recurs by just adding this tail W annotation to it for me that's actually a very useful fallback uh it's true that if you approach function programming then often the nicer way to express yourself is with combinator libraries you use map flat map filter collect things like that and you combine them all and in fact that's the right way to go about if your if it fits your problem well but there's always a time when sometimes you run out of operators you run out of combinators your problem doesn't really correspond to any combination of these things or any any natural combination of these things and there are sometimes also times where you need to be sure you have squeezed out the last cycle of performance of your program and in both of these cases this recursion and is actually a very good fallback so tail recursive functions can essentially Implement arbitrary State machines very very efficiently so they it's good to have that in the back pocket so to speak to say well if I must if my if my combinators are not good that for me is the fallback uh where I know that I can always go back to that and do something that works and is efficient even though it might be less elegant and sometimes longer than the combinator approach but speaking of combinators that they are of course also very important and the combinators in particular very powerful because they let me abstract computations into values so I can have a a method like filter it takes a parameter of what I want to do so I abstract out the operation with which I filter into a function value and function values of course are very important are backbone of functional languages and by now they are so common that basically every language has them except of a few holdouts uh which uh which are the minority of even mainstream languages almost all languages have them Java 8 has now even function values and and that's a good thing I think it's it's excellent that we can agree on that now everybody so number six would be aggregate so uh if you have data then of course we want to have some way to treat these D put these data treat these data collectively put them in a collection and the importance in the functional world is that once we have these collections we want to transform them we don't want to do create read update delete because that's imperative that's really element by element so the essence of functional programming is transform immutable data and do the same thing for collections and these Transformations are particularly Pleasant I believe in Scala because they're very uniform so there are rather large but uniform sets of operators that lets you transform Collections and because they uniform these operators they're the same for every collection they end up being rather easy to use because uh you can transfer the knowledge that you know from one collection let's say from lists to another collection you can go from lists to arrays or to set or to some distributed maps and so on it always it's the same operators that apply it's a very powerful principle so what what you have here it's just a little works worksheet transcript I don't know whether it's so legible so uh we have here an array of persons and then you can map with a function underscore do name over the persons you get the names of these persons in the array but you can also map over a set in this case we map with the division by two operator and the new set will then have one element less than the original set so that tells you that no sets are not sequences they behave differently of course because you eliminate duplicate but they both have a map and map does a natural thing both for sequences and sets and even maps have a map so here you have a map over a map that uh inverts the original map so you can do lots of stuff and that's not just for the basic types but it also works in sequential and in parallel and it works on a single computer or it works on a distributed system using a software such as spark okay so I believe actually collections are a big argument in favor of Scala they work very very well but not everybody agrees with that so I thought I would just uh respond a little bit to the collections to the objections that you have heard sometimes on collections so one argument that uh you hear sometimes is that that people say the type of map is ugly or a lie well to explain that let's look up the scalar doc so the doc official documentation of maps and here I look look up the one for arrays so here here's what I see so map takes a type parameter B and the function from A to B and it gives you back an array of B looks looks reasonable right um but there's this ominous thing which says use case so what this does says is in principle if I am a client of an array then that's the type I need to know that's how arrays work from for me as a client perspective but what actually happens is that if you look up the implementation of map you won't find it on array you will find it in a much more General space where it works for all collections and that's very important because uh we introduced the scheme in Scala 2.8 uh four years back five years back uh what's important is that before every collection had their own map or their own implementations and that led to a lot of discrepancies discrepancy creep between collections because uh some of these implementations might have been buggy others were correct so the more implementations you have the more the more uh scope is there for errors to creep in and what's more than there might have S subtly different semantics for instance there could have been a take where one implementation would just truncate the the the argument if you take more elements than the collection is is long it would just reduce and take the whole collection the other implementation would be different and maybe throw an exception so you had and furthermore you even would have sometimes one operation like append on one collection and on the other collection it was called concat and these are all these things which are friction these make things hard to use you don't have the same operations the operations have subtly different meaning that's the last thing you want to have so what we decided would be much better is if there was a single implementation of single reference implement ation of every method you have in collection sometimes that reference implementation is overwritten in particular collections for performance and that's okay but there should be a single fallback a single reference implementation now if you look up that type then uh you get this uh tip down here and uh there was a uh that it does indeed look pretty horrible so it now has two type parameters B and that and uh here's the function that goes from T to B and then there's this implicit parameter which is called a can buil from and it says well if you have an array of T and an element type B and you can build a type of a type of a collection of type that then that's what you return yeah that's that's complicated and uh I guess uh the good good point is that almost no programmer ever sees this thing until they look because uh that's something that is very carefully hidden in the compiler but you could ask well why why does it have to be so complicated why can't we do it like this so in in a sense you could say things that have maps are funs and one idea to treat that would be to say well here's a funter and it has an higher kind type f and an element type D and then map would just uh go from take a function from T to U and give you back the same fun but on a new element Type U and in fact when when I started to work on the new collection designs the reason why I did that the reason why I started at all was that Scala had higher kind of types so I thought I could do something like this and it would be fantastic it would would be great that was the otherwise I would never have got gotten into collections unfortunately it doesn't work why does it not work well think about arrays so uh in arrays if you have a type of map if you want to build a new array what do we have to do we have to give we have to provide a class tag because the jvm demands that for arrays to build an an object of of an array I have to know the element type statically I have to put that in a class tag and there's no way I could put a class tag in the signature it doesn't let me add a class tag it's too pure for that and it turns out that the class tags for arrays are just one example there are many others uh for instance for an ordered set i' need a comparison operation to essentially eliminate the duplicate and order order well and other collections have similar constraints that say well in addition to the pure type of map I need to know something else and these need to know something else that's precisely what's always encapsulated and it can build from so now you know why there is can buildt from and why it's necessary to make it make a collection system that is very very general uh if we didn't do that then essentially we would have a collection that A system that only worked with a very very small subsets of of of of collections that uh for the end user would be weird to say well why can't I have ordered sets why can't I have arrays why are these things not collections there should be and I think that's we we just have to work a little bit harder to make them collections okay so the other collection I just uh the other uh objection maybe I just skip uh you can ask me afterwards if you want to and then I will talk about that so number seven in the uh um uh list of things to do with uh programs is mutate and I put it as last because I think it's the one we should be most careful about but we shouldn't forget about it all together so mutating uh variables is something which is an essential part of programming and uh what I would recommend is that we minimize that but I I'm not recommend that we forget about that that we don't use it at all because while Global State can often heat to lead to Hidden dependencies and that's not good it can also cut down on boiler plate and increase efficiency in Clarity when it's used widely so what I did is I took a u a survey of the code that I wrote uh recently in the new compiler project that I did and said well why does that use mutable State and in fact it's uh done pretty carefully so uh where it does use multiple state are for these five uh purposes uh where essentially three of them are only optimizations so the idea is what we want to do is we want to write a compiler that is very functional that has a predominantly functional architecture and then we want to make it very very fast hopefully much faster than the current Scala C compiler so how do you make a purely functional program or almost purely functional program very very fast well you have to pull some tricks so one trick you can pull is very aggressive caching that's something that essentially function programs give you uh you have a function and the function will give you the same result every time you apply it to the same arguments now if the function gets applied repeatedly to the same arguments and the function is expensive to compute you might actually be better off by caching the first application of the function result and then reusing that result later on and but caching of course is an art so you can't rely on just a built-in thing in the language in fact if I look at what the compiler does we have a large number of different forms of caches there are the lazy vales that the language provides but there also memorized functions that are backed by a hashmap or we cash map they interned names and then they even lru caches because we need to bound the size of the cache so that's that's caches of size eight or 16 that essentially hit the last 16 member calls on a given type and return you one if if it's one of the the last 16 so caching is an art and it requires the application of mutable state in a clever way but caching done well is not visible for the clients of the thing so the compiler would would run just as well albite much more slowly if we turned off all the caches in fact we have an option to turn off the caches and verify that fact uh then there's persisting so once a value in a map is stable uh we can actually transfer it as a into a field of the object itself that would clear the value out in the map and therefore essentially again avoid memory leaks and improve the access time to that value we do that uh at at at one point in the compiler when we talk about the instance of a of a type variable once the instance is determined we can put it right in the type variable then there there's a trick on copy on right where we say we we have trees and the syntax trees again they're purely functional but under the cover they have Essen a field for the types and you essentially you have one right free you can use reuse the same IM to untyped tree and you can put one type in there but when you come back and says well this untyped tree in the new context has a different type and you can't of course overwrite you have to essentially copy and write the tree and finally you have fresh values where you have to generate in a compiler often fresh names so a unique ID or a unique unique name or things like that and of course for that you have a global counter and you bump it up every time you need a new new name that's what everybody does so what I'm saying here is all these four things are really your use state but for the users of your program it's not really visible except for the last Point unique values you can argue about that it is visible but it's generally uh it doesn't matter what what name it is so it doesn't so so so you don't need to do it purely functional and if you do all that then the compiler actually was left with two variables uh one that kept the current constraint where the constraint is again an immutable object and one which kept the current Diagnostics which is essentially the error lock and those two uh variables we have to version them that means that there's actually a sort of backtrack search where they could be could have different things and you you need to be able to backtrack and um and keep multiple versions of these things around okay so now you could say well essentially typing means um um here's a function typed uh so it uh takes an untyped tree and an expected type and gives you back a type tree but in addition it also might manipulate these two variables under the cover namely the it might add to the constraint and it might produce errors and here's another method is subtype that checks whether one type is a subtype of the other but again in doing so it might add to your constraint set that's essentially an implicit uh documentation that you say it can do the things so if I were completely pure I would say at this point well for these things you should use a monard right who here knows what knows not what monads are everybody excellent okay so a monad is uh essentially a thing where we say well where I have this result type tree here here I wrap the type Tree in something I call a typer state so I say I return at hyper State that's these two pieces of State the constraint the and the and the log and I pair it with my actual result the type tree and I do the same thing with the subtype so that would be the pure approach to function programming which says we shouldn't have state you should or if you have state then you should simulate it with these monets uh but if you if you look at the use usage examples then what was this would become that so here's this very simple line which just says uh if T1 is a subtype of T2 and T2 is a subtype of T3 then return some result if uh is subtype returned not a Boolean but a typ of state of Boolean of course you can't do that what you would have to do is use a four expression or a series of flat Maps which is the same thing so the four expression would read like this it would say well if for C1 taken from the result is subtype T1 T2 C2 taken from the result is subtype T2 T3 if the condition C1 and C2 is true then yield the result so there's a systematic way to com to essentially convert the functional programs into this monatic world but you have to ask yourself have we really achieved something that way is that really better it sure it's longer it's probably less efficient too so so what what have we achieved with that so for me actually uh the important part of this thing was that by thinking really functionally we have reduced the essential state to two vars that that's that's that's the big thing that and I think that's the thing that we should try to to to achieve think very carefully about the state minimize it to the absolute essential bits whether then with the remaining State we actually have a type that reflects that state as in monuts or we leave that essentially implicit is a secondary concern that's a concern where we essentially we have a tradeoff a trade-off that we can make in either directions and in the compil definitely the trade-off is against the monards because essentially it's pretty clear what kind of operations affect what kind of State uh essentially everything in the typer can produce errors and uh add to the to the constraint set and everything in subtyping can can add to the constraint set and that's it so what I think here is that these tradeoffs these typing questions are really more pragmatic than fundamental so here here are five fine programming languages and in a sense none of them is right in their approach to typing and that means that essentially everybody is is is is correct so if I program in a language like closure I don't care about types at all all I want the compiler to do is check my syntax and I said the types are implicit if I programming a language like Scala then I have a more traditional approach to static typing so the types verify what goes into my functions what comes out of them and maybe the the types of variables as well if I have a language like hcll then in addition to that I let my types track my effects like the typ state uh if I have a language like RIS and in addition my types actually tell me even more about my values if two values are equal or if certain predicates hold about my value that's these dependently type programming languages and if I have a language like then essentially my types give me a total correctness proof of my program which is would is of course the best only that except that it's very very hard to achieve you have to work really hard to get that far uh so what I what I'm saying is in that Spectrum the question what you subject to types is one which is pretty pragmatic and there's no right answer because if there was a right answer you would say well four of these languages got it wrong and I think that would be that would be too presumptuous to do to to say that so all I'm saying is it's all a question of trade-offs good so to summarize the simple parts that I have identified in Scala there each come essentially from a fundamental action we compose Expressions we Nest Scopes we match with patterns we recurse with recursive functions we abstract functions into values we aggregate Collections and we mutate variables that for me is it and if I look at my code and I say well that's basically it there may be some odds and ends like lazy vs that hang here and there but they're not essential to the core of programming that's essentially what what it's about except then of course with these simple Parts you build modules so let me just finish and talking a little bit about that so what is a module well actually a module can take a really large number of forms a module can be just a single function uh module can be an object a module can be a class so an object that gets parameterized an actor a stream transform or a microservice or any of probably more uh instances the importance of a module is that the focus is on how a module can be combined with others and not so much on what what what it does in isolation so that's what makes a module a module is something that can be combined well and what's important is that since Scala is a statically typed language in Scala these modules they need to talk about values but also about types because a type is an essential part of the interface okay so let me just skip that in the interest of time so the features that Scala has for modular programming is I think first uh we need a vocabulary what goes in and out of a module and there I think it's precisely the rich types with static type checking and functional semantics that we have talked about so far that we can build up these types with case classes that we can work with them functionally that we have a functional theory on these things um then the next thing is uh pretty mundane we the simplest form of modules are objects so an object is just something which is a container for functionality methods typically uh we can parameterize these objects uh then they become classes so once an object takes parameters it becomes a class and we can create an arbitrary number of instances of these things so that's sometimes useful if your module is just not a unique thing so what you want essentially a dynamically varying number of them and finally TR that's essentially a mix in in another dimension so a trait is essentially a slice of behavior that exists to be mixed in in some other Behavior so you have classes and traits which is essentially parameterization and mixing slices that's number four so I was going quickly for the rest I'm going a bit more slowly so the next thing is abstraction so abstraction means uh taking a concept and giving it some name and afterwards using only the name and the direct form of abstraction is directly by name object or programming gives us that so we can actually have in a trait or in a class things which we which to which we give a name but not an implementation what they mean and uh because modules in Scala talk about types as well as values we can do that for types as well as methods or Fields so here is an example uh where we uh treat a problem which is uh hard to do in general and that's how to write a library for graphs uh without overc committing ourselves at any point to some particular form of graph whether it's labeled or weighted or what not uh and to any particular implementation of graphs whether we do that with a metrix or a an edge list or something else so what I start with here is to say well I have a trait graphs and it has nodes and edges and I don't know what nodes and edges are but I know that there's a type of nodes and there's a type of edges that's all I write here I don't need to write more the other thing I I I know is that there is a a function predecessor that takes an edge and gives you its note and there's a function successor that does does the same thing and then I have a graph U over these given alphabet of nodes and edges and um a graph is a um a type where I know that it conforms to this graph signature I say what what's in a graph a graph has a set of nodes a set of edges for each node there's a set of outgoing edges for each note there's a set of incoming edges and there's a set of sources I could say these two are fundamental that's what defines a graph and these three here in a sense utility methods that are very useful to have on graphs okay so so far it's pure interface there's not a single bit of implementation every anywhere I just told you essentially What's the vocabulary when we talk about graphs there are things that are nodes and edges and how do they hang together and uh what what do I what operations do I choose to have in a graph okay so let's work on the graph thing thing so independently of what our nodes and edges is we can give you a possible implementation of a graph and here the implementation is the most concise possible and that probably also makes it the most INE inefficient one possible so here the graph just would get the nodes and edges as two sets as parameters and then the outgoing edges of a given node is just I go through all the edges and I filter out those edges that have the note as the predecessor the incoming edges I filter all edges uh to to those that have the node as their successor and the sources I filter all the nodes that don't have any incoming Edge that's the sources of the graph okay so what's interesting here is that we can at any point Define what we know and leave abstract what we don't so here we've what we have defined now is this thing graph which so far was just a promise we said well there is a type that satisfies these methods and here we have said well here it is that's the class graph that that that that that we produce here but on the other hand here we don't really know yet what nodes or edges are so we just leave them open it's completely okay in this objectoriented class-based approach to just say well if you don't know something leave it abstract okay so finally here we actually give a concrete implementation let's say the graph is a social graph so my notes are some persons and it's a simple social graph so my edges are just pairs of persons and they don't have a weight or something else so it's just this thing here these two type definitions and then I also need to implement these two methods the successor and predecessor methods and I'm done so then now I can mod mix the concrete model that explains what the nodes and edges are with my abstract graph model and I have something which is a complete application so what's interesting here is that uh there actually two things that are two sides of the coin that are in a sense exactly the same thing namely encapsulation and parameterization and that's rather surprising so if I look at the first example here you could say that's an example of encapsulation i' I'm hiding the type of noes from you and the type of edges you as a user of graphs here on this level I don't know what nodes and edges are so that's a sign of encapsulation I can't mess with the internals of node no nodes and edges on this level here I know what they are but that that's the implementation level of the graph and the at the same time by these definitions I parameterize the abstraction so I provide the actual param argument for the abstract note and and and Edge parameter so parameterizing and abstraction using this by name overwriting and encapsulation are two sides of the coin depending on where I look at whether I look at the abstract class or the concrete implementation okay so that was this name-based abstraction but there's another one and that's by position and that's much better known normally that's just plain parameterization so here I just look at the types so of course in Scala we can parameterize classes and rates so we have the class list of T and set of T and then we have this uh thing called variance that some of the parameterizations are covariant and others are invariant and others are yet others are contravariant so that's sort of something that often people find more complicated so when I when my son was first um looking at Scala and reading the programming Scala book he found that very easy so one chapter after another no problem no problem until he got to Varian this is really weird this is a weird thing in Scala and I had to sort of explain to him why why that is and it is something that trips up a lot of people so what variance of course does is it says well if I have a list of apples is that a subtype of a list of fruit and here the answer is yes a list of apples is a subtype of a list of fruit because I have written this plus here the plus makes a list of apples a sub type of list of fruit for set I have chosen to leave out the Plus that means a set of apples is not the subtype of a set of fruit because the subtyping thing doesn't go in the in the in the set Constructor so the the the sub type relation I have when I write that but not when I write that so that's a hurdle to overcome when I do abstract by position and but what's interesting is actually that there is a very tight correspondence between abstract by position and Abstract by name and furthermore that correspondence actually explains what variance is so it's actually rather nice so we can model these parameterized types as follows we can say well let's let's find an encoding where for every parameter I encode that as an abstract member and for every argument I encode it as a refinement so we'll see how that work works so where I had before a class set of t with something in there now I just write a class set unparameterized but instead of the parameter I give you a type memory I just write it dollar T here to say well it's not the same thing it's not the the name t by itself but some other name that the compiler generates okay and set of string would be the then the type set but now I have to declare that the type T the type parameter T is string so I just write it right here type Dollar Tree T equals string so here you could say well I can see this as just essentially a syntactic sugar a shorthand form for that that's rather neat so let's see how it works with variance so let's now look at list so the class list which is covariant would be expanded in exactly the same way but then a list of number would be expanded slightly differently because a list of number I say well it's a list and the type T in the list it's some type which is a subtype of number or otherwise put a list of fruit would be a list where the type T is a subtype of fruit and that makes a list of apples a subtype of a list of fruit because apple is a subtype of fruit so the type T could be apple here and it would still be correct so the question is whether I have an equal here or a upper bounded by that's the trick that makes variance Works work and Contra variance then would translate into a lower bound here but that's maybe one step too far to explain here okay good uh number seven and last uh I think for for the essential modularity features in Scala is uh implicits to keep boilerplate uh away so keep boiler plate implicit implicit parameters are a the concept that helps you cut quite a lot of boiler plate and get a lot of flexibility in your program so they're surprisingly versatile the original use of implicit parameters was essentially as a more modular version of a type class as you know that in hll so typical example here is the minimum method that takes that can work for any arguments of type A as long as there is an ordering over this a so there must be an ordering that explains how to compare elements of type A and the idea is just you need to get that from somewhere uh uh and uh you don't care from where it comes it just must be unique there must be one most specific ordering that you could have and that's done with this implicit here so you could of course have written the ordering explicitly could just demand the ordering as a parameter but it would be very cumbersome to for every Min to have to pass on ordering in this minimal me method because probably in most of your program you deal with the same type so you're passing the same thing a 100 times to your Min methods and the other methods and that's something that you typically want to leave implicit that's why the parameter is implicit okay but there actually also other uh applications that are equally powerful so implicits can represent a context so in the compiler for instance we have this thing that essentially everything you need to know from anywhere it's all in this object which is the context the context in which you are and these contexts they are typically rather stable they're the same in large parts of programs but at some points we say well in some specific part my context would be different and the context would then evolve so so sort of a tree of contexts that get passed into these things and again basically everything needs a context I could pass it everywhere but it would be extremely cumbersome code would look horrible if every operation would have would need an additional context parameter a related thing here is a a um the default options so that's just a an argument where uh normally to the compiler I pass a certain string of options in a testing framework and that's sort of there a there's a default and I want to override this sometimes I want to override sometimes the options are different and there again uh implicit parameters are a very useful way to do that if you take a step back and squint your eyes a little bit then that's really what dependency injection is that's things that you sort of have to pass to everything in your program and you want some control over what it is uh but you don't want to have the boiler plate to pass everything everywhere explicitly so implicit parameters give you dependency injection they're very nice way to do that and finally implicit parameters also give you capabilities so here's an operation that says for instance access profile of a customer ID and you can access a profile of a customer only if you have admin rights so that means for the the capability uh to enable access profile I need to pass my admin rights as a as a additional parameter into this method here so that's a classical way to do capabilities in an objectoriented way a capability is just an unforgeable object that I allocate somewhere and then I pass it to everyone who needs it and again it's very very tedious to do that explicitly because in a situation like that all that matters is I call somebody's method that needs a capability and I have it how this cap capability then manages to get pass to the point of call is completely irrelevant I have the capability you need it I can give it to you so again implicit there a there's a great argument for implicits here so to uh conclude with the module part so I think the the things that matter here are a rich static type system objects give you Atomic modules classes give you parameterized modules traces give you slices of behavior you can abstract by name or by position but the two are really sort of codable you can code abstracting by position into abstracting by name and you can cut boiler plate with implicit parameters again that for me is sort of the vocabulary that I use every day to write modules in libraries so in summary I would say it's a fairly modest some even say boring set of parts that can be combined in rather flexible ways and the cover of course is always that's that's my selection not everybody needs to agree um before we finish maybe let me just have a quick slide on the other parts that didn't end up in this list and there are quite a few of them also in Scala so here are some things that are not on this list and they're not on the list because they're either not as simple or they don't work as seamlessly with the core as the other parts that I've I've given so the first one would be implicit conversions so while I was singing the Praises of implicit parameters the same is not at all true of implicit conversions implicit conversions just say I have a type A and then under the covers it gets converted into this type B whenever necessary uh the problem with that is that the more of these things you have the the stranger there's the interaction between them so I guess implicit conversions they are the seductive thing that it it starts out great if you have one or two and then you add three or four or five and suddenly it doesn't work that great anymore so it's a slippery slide so that's why I would say well stay away from implicit conversions as much as possible uh second thing is existential types which are sort of a promise to paper over certain aspects of types but it typically doesn't work very well with inference and other things also structural types can become very complicated also inefficient because they sometimes require reflective access higher kinded types that's something which um um I I think in the end we need but we I would like to I'm a bit frustrated that right now it's not integrated very well with the other types and the and the uh type inference in particular higher kind of types don't really have a role in this duality of type parameters and type fields that I have shown you and finally macros which of course are an amazingly powerful Hammer that can do a lot but they're also very complicated so what I should say is that all of these things are under language feature Flags or experimental Flags so to enable any of these five things you need to turn on an explicit option or you need to have an import import language do existentials or import import language do structural or things like that so that should make it clear that they are outside de core you could ask well why do we have them if they're outside de core well I think part of the reason is that uh Scala is really not sort of a monolithic uh industry uh company language a company language you can work for for years be behind a wall and then you throw it over and you say that's it and there's nothing else Scala is a community project so a lot of people working on Scala lots of different ideas and that's something that means it's much more fluid than a corporate language and we need to manage this fluidity in one way or the other so what we have chosen to manage it with is precisely this mechanism to say well we let things in but we label them experimental or we label them in a special language feature flag or things like that and that means that you can you sort of are warned you are you you have to declare explicitly that you now want this particular piece of the language so my advice actually would be to if you want to keep things simple I would avoid these things mostly unless you have a clear use case and of course one of the use cases could be that you want to use Scala as a domain specific language as a host for a domain specific language or another language and that's a perfectly valid use case and then you might find some of these things really essential but if what you're after is really bright just some program some project with just plain simple Scala then I would say I've shown you the parts that I think qualify thank you happy to answer questions yeah one of the things was hard in learning scholar coming from java was does it look like Java do I try and make it look like H there's so many ways of doing things and I'm finding my way but if I knew a preferred way to go from a Java ADM to a scholar idiom it was like well when you're doing this in Java try and do this in scholar that's way and there's some of that with region yeah I didn't find enough okay so what's yeah what is good advice to come over from let's say Java to Scala because there's a confusing number of different possibilities you can write it like like a Java without semicolons or you can go all the way to simulate hcll on the jvm yeah um I I I would say to well first advice is keep things as simple as possible and then I think the you you have here the building blocks that you say well why I Ed the loop uh then what I would say well let's consider first do I actually have a combinator that does what the loop did before like is it a map is it a filter is it a fold and these things you sort of have to learn gradually because there are quite a few of them and you wouldn't of course know initially all of them but some of them you know and some of them you can apply directly and then your Loop has shrunk into a single expression and that's a win then the next step would be to say well uh if it's not that then let me just write a recursive function with the same thing and that's again it's sort of a a a thing that you have to do a couple of times to get the hang of it how it works but uh once you got the hang of it it becomes quite natural and I think with these things that's sort of already it uh there's not that much more to do uh you probably want to uh move statements into Expressions uh you want to keep uh minimize mutable variables and keep them immutable and things like that the important thing also I think it is we have to realize it is a gradual process so we can't assume that the code will be good the first time that's just impossible and it's not just impossible for somebody coming from java but I think it's it's a general situation for all of us anytime that the code we write after a year we might have written the code differently we we learning constantly and that's good that's okay right so so I think the important thing is not to be afraid to write code that maybe is not perfect just write it and uh if you come back in a year and want to rewrite it and do it better then that's also perfectly fine the backit SCH I want to ask more in terms of like conr programming I know that one of the benefits of functional programming is it makes easy to write convert software and I I love Java uh but I I feel like conen is not easily accompl job you have to manage States and get a lot of bus um how what can SCA offer programing and second question is can you make can you run Scala for and applications okay first question was uh what has Scala to offer for concurrent applications uh actually has to offer quite a lot uh so uh there's a there's a very solid and refined set of libraries that deals with concurrency uh starting with Futures uh going to act AA actors uh going to reactive streams I think that we have the right abstractions to deal with that so you then you could ask well why are there the right abstractions why do they make concurrency easier and I think the answer is precisely that uh if in a concurrent setting you want to be as immutable as possible because every piece of mutable state has to be protected by a lock and is a piece of liability so the more things are immutable the the the better and then the other thing is you typically need rather sophisticated abstractions and they profit greatly if you can pass a a a a function as a parameter so for instance if I take Futures as a as a very simple example so the traditional Java Futures they they block so you have a future and then you you essentially you set up a future which says at some point later on this thing will have a value there's a threat that computes it at at current and the consumer then can go to the Future and says well now I need the future and at that point I block whereas in Scala Futures you have this uh method called oncomplete which says well if I have a future then when the future completes then call this code which essentially does the continuation that continues dealing with the future and that's this this sort of reactive approach to to concurrent software is everywhere and to make it work well you really have to have a lot of the functional idioms so I we we were giving a Kera course that explains that better it's called principles of reactive programming it was taught by myself and Eric Meyer where Eric talked about essentially observables and Def Futures uh Eric is the one who did uh Java Java RX and RX extensions on net so essentially everything reactive he plays a big hand and then there was Roland who uh presented AKA and actor and I think if you see that course then you see that really to make reactive do well you need the functional abstractions because otherwise it's it's it's too hard uh the the abstractions are too clumsy and and you won't do a good job that was the first answer the second question was uh does Scala run on Android and the answer is yes it runs on Android there's a tool chain that you can use uh I think it's integrated in in the idees you can use eclipse or intellig and put use Android applications yes most of have the luxury of starting from scratch if we want to merge scal into our existing systems a lot of us have Java systems um so my question is about framewor work with I know there's for Scola I haven't used it that much so I don't know L features I'm looking at grad for other reasons and I'd like to know so what the you know is love between those two Frameworks what is there um you mean for build Frameworks spt and gradal yeah okay so so there was a question I think the general question was uh how do I um combine Java and Scala what libraries and Frameworks exist that work with both and in particular question was about build tools so it's definitely possible to build Scala with grader uh the uh there's the one thing where for efficiency we have this incremental build uh which is essentially uh currently integrated in spbt but it's also available in a separate component called zinc that can be integrated in Gradle and in Maven so I would say the the the best incremental build is currently linked tied to spt but there is a way to build incrementally that's also available from Gradle and Maven and uh we we would like to make that more widely available so even even the best incremental build for the other builders I mean definitely we are not I think the scalar Community as a whole and type safe in particular is definitely not of the position to say it should be spt and nothing else if you want to build with Maven and Gradle that's perfectly perfectly good as well we're very agnostic there uh in the other Frameworks it's very similar so what types saave does the the AKA framework and the play framework they both essentially dual usage so they have a Java API and they have a scalar API and uh with Java 8 we're actually very excited because we think that we can Java 8 will help us get the Java apis to improve the Java apis and get them up to the level where the Scala apis are so it's definitely a a program that we have that we want to be useful for just as well as scalar scalar clients yes after sorry first first then work on improving compiler speed is there something developers can do to help the compiler aome um yeah so talking about compiler speed so I we did try to improve it but what can we do now to essentially get faster builds so it's interesting to see why why are the builds so rela relatively slow so first the question is how slow are they uh what what I get typically on on the MacBook Pro on a on an iMac is somewhere between 500 maybe thousand lines a second that's sort of what I'm what I'm what I'm getting when I do a clean compile and um that's not super super slow but it's definitely not as fast as let's say Java that gets in the tens of thousands of lines of seconds so so why is there this difference in speed so the problem is there's several factors that all multiply with each other it's not just additive it's multiplicative the first Factor there is is that type inference is costly so type inference means that the compiler has at some point to do a search in particular implicits so it might search many things which then don't pan out and that's that's wasted work and that's that's reduced work so to help the compiler along generally the more types you annotate the better because that means the the compiler doesn't have to figure them out and also uh the fewer implicits you have in scope the better so if you have a lot of implicits visible in a lot of things then that's generally bad because the compiler has to search them all so that's sort of a hint what to do there uh the the other thing that uh costs a speed is that the scalar compiler has to go through a large sequence of Transformations which all lead again to type trees and while it does these Transformations the code becomes a lot bigger and that has particular to do with uh with with function values with lambdas because currently each Lambda gets translated into an inner class and the inner class then gets translated into a top level class because the jvm doesn't know inner classes and there's a a lot of duplication then in all these classes both for the code generation and internally for the type checking so that means that at the end the scalar program is much bigger and it's slower to type check so then yeah of course there's a lot of time that the compil takes to go through that hopefully oh fortunately that actually that part actually will improve uh uh the next version of Scala will Target Java 8 and Java 8 has lambdas so we don't need to generate these inner classes anymore for all the closures that we have in fact that's that will be done at at program load time uh because the jvm jdk8 will actually generate these Lambda classes with with invoke Dynamic when the program starts so the compiler doesn't have to generate them anymore and that should definitely help keeping the whole code size smaller and therefore also keeping increasing the speed of the compiler yes so I came to from the world wanted to know what you think about so difference or similarities between Scala and FP so uh Scot and F are indeed quite have have quite similar roles uh they we are sort of on different platforms f is on net we are on the Jo jvm uh I I'm very good friends with Don s the creator of FSH we actually had quite a few discussions and uh some some of the developments have have been actually quite parallel for instance uh in around 2006 or seven Scala had a thing called extractors which is essentially extensible pattern matching and F sharp had active pattern PS which was a version which which which is a variant of that which is very close to what we have in extractors so some of the things actually there was sort of cross polination between Scala and FP uh in the general Outlook I think there is some difference uh I've said Scala is a modular language and part of the modular genes come from object orientation we talk about objects classes traits and it innovates a lot in that space whereas f is traditionally a functional language in the ml style so essentially an O camel but the modules got removed and the net object system just got integrated directly Lock Stock and Barrel so the FP idea was we let's let us take a functional language and let's take just the objects of the host platform because we don't care about objects and and make that work so that's sort of the difference in Outlook why some people say more scal is a language that comes from object programming and adds functional programming and F is a little bit the the opposite but we we we meet somewhere pretty close in the middle I guess yeah uh you were asking yeah so there might not be as well advertised regarding to type safety so one of the other sold maybe not advertis the actor system Scala which is actually inherently TI less yeah um can you speak a little bit about so why is the actor system typeless um yeah that's a good question um the so maybe one if I go go back to this slide here that explains it a little bit yeah that's one here so the the type system of Scala actually is not so much about to capture a lot of properties with types it's more about to say well uh we don't want to be embarrassed towards the people inclosure anymore how clunky our type system is how clunky our types are so the so the types so a closure guy would typically say I would never program in a language like Java because the types are horrible so so the the the main purpose of Scala types was we want types that don't suck right we we want types that that we can we can defend towards towards a dyamic Ty DY for the moment not necessarily to capture a lot of properties that's actually what I think might happen in the future we might be able to actually move this a little bit over there to capture things like effects I don't think we'll do it with monuts I think there some things that still wait to be invented but I have high hopes that actually there will be movement over the next years towards that goal but I would say Scala is a fairly conser conservative language says well we want types that don't suck and we don't really want to go all the way over here where these things might become very complicated and inated in a pain in the butt of the Developers for the actors now the point with the actors is that an actor you send messages to an actor but you send more than one so you you send different kinds of messages it's not always the same the messages have different types U you could of course say well create an algebraic type or class hierarchy that defines the messages and send them and maybe that would would have been the right thing but it's probably very hard to do that because different actors at different times take different messages so the type system would be an overhead but it still wouldn't be very accurate you can go further and that would be then really interesting to say well what I actually want is more that the types give me the protocol of the actor for instance it could say the actor always expects first a message of this kind and then a message of that that kind and it will or it will always respond with an act uh if if there is a is is is a certain request or things like that and these type systems exist also they're called session types they're very interesting but they're so far quite quite uh experimental and researchy so again that's sort of something where we consciously said well let's do the safe thing let's do essentially have untyped actors because we don't really know what a good type system for them is yet and the other part the the other reason why that works actually pretty well is that because of the way we treat pattern matching in in Scala patterns recover the types so you can put in an any or an object in the message and then you say well let let me match whether it's some this this message Constructor and that message Constructor if the match succeeds I know exactly what the type is so in that sense this sort of small amount of dynamic typing from the message that comes into the patterns is uh it definitely I mean there is there is you you you might want to tighten that up but for me it's not super urgent and that's that's why sort of the the the compromise was let's do it as an airline treat them dynamically what questions for the live stream okay great yeah let's switch to the live stream sure uh the first question is from Ted Fujimoto uh how rigorous are the evaluations used to decide user friendly changes to Scola how rigorous are the evaluations oh well it's a community process so uh that means that if we have a propos a change to Scala then it gets proposed on the mailing lists and I would invite everyone of you to contribute on these mailing lists uh there's a more formal process with the Scala improvement process that means there's a mailing list for that and then at some point if it's something bigger then there's a Scala Improvement proposal and that gets discussed at length and then there's a there's a jury that votes on that one uh but there is no we don't do formal usability studies or anything like that I don't think we have the bandwidth for that so essentially what we need to do is we need to rely on taste and user feedback um next question comes from Brian McKenna coherence is a whole point of type classes why would giving that up for modularity be anything like a sensible idea hi Brian yeah I heard that argument from Edward KET uh repeat the question okay the the question was coherence is the whole essence of type classes why would giving that up for modularity be sensible so what coherence means uh is precisely the thing that for a given type class I have one and only one possible implementation of the type class which is antim modular because one and only one is means it's a global name space so why is that a reasonable thing to do so I think it's precisely because if you need coherence then you can work around it so what I would do with coherence I would put the thing essentially the type class and its usages into an enclosing object and that could could be made to work I um have have seen the argument from Edward met and I would like to really have the time to meet him sometimes and discuss that with him because I think it's a it's a straw man it's a thing that you can easily work around for instance by saying with the ordering argument I would just say to say well I let me have an ordering object that contains this ordering and it is this object and then everybody else has to implicitly depend on this ordering I have to uh essentially parameterize my system with that ordering and Scala has a lot of parameterization methods I've shown you the abstract types and the abstract fields in in classes that that let let you do that so I think it's not necessarily a conflict between the two but of course you have to acknowledge the fact that in a modular system you can't have a prior a globally unique instance so you have to arrange things that that's what you get next question comes from Ian um is D going to uh is do going to be the next Scola compiler or will it remain as experimental thing is do going to be the next scolar compiler or will it be experimental um it's currently too early to tell uh I actually have um I'm pretty confident that it will most of the Doty ideas will find their way into the Scala code base uh uh in what form whether essentially they will be selectively backported into this Scala C base or it's some point we'll switch over I think it's it's too early to tell but I would say most of the of the ideas both in compiler design and in language design of Doty will will find their way into SCA I think that so far that I'm I'm confident by now so far from what what we've seen so far that that will be the case next question James uh I haven't seen enough of that and I I I guess there it could be in principle yes but I don't before I say say something with a precise opinion I would have to know the water remover better so don't nail me down to something but definitely the idea of having a lint tool or a style Checker is something that is very good that is very important in this whole context and that's actually something we're working on so in the 212 uh time frame what we want to come out is a lint Tool uh that um essentially does a style check of the code and uh sniffs out some other cold smells and what's also important is that this lint tool should be configurable so that you should add your be able to add your own rules or remove selected rules that you think shouldn't apply to your code base so I think that's that will be once once we have that that will be a very important contribution and water remover is sort of one of the prototypes we look at here I have a question about scolar community so uh we did a panel on Scola in 2018 with r Johnson following his keynote and we had five uh voices from scolar Community ranging from kind of mild to spicy uh and in terms of functional and so hor then of first we know at the stripe he came up with this idea of Scola tribes right so essentially Scola is leads to tribe so there is a Twitter tribe is Middle the road there is a beginning tribe which is Java converts and there is kind of FP extreme tribe which which basically is like crypto hosers uh using kind of Scala you know where they prefers Hill right so question first like Community question how do we keep this community together what are your tips to keep the community together and so second question is practical let's say your review of engineering there is a lot of reviews of engineering and C in this up and is one of them I don't know if you have this situation but let's say you have two employees right one is kind of new convert he use lots of bars and it doesn't see the point of vals and things and the other is kind of a hill guy who you know Works in scholar and he really wants to move it all the way to Hill so the new guy does a commit with a VAR and the the hill guy cancels it right so the hill guy would not allow any vs into the code base the the new guy doesn't understand why he can't have his counter for global object right so how do you first of all as a kind of manager uh manage this in reality and secondly how do we keep these tribes from splitting apart in in the community yeah okay so the question was we have all these Stripes how you manage to keep them from splitting apart how do you manage dis agreements between the different dialects that we have on top of Scala I think that my talk sort of tried to address that in to some degree to I said well first we have this aspect of a growable language that Sprouts also sort of sorts of dialects and those are your tribes and uh we just have to be clear to say well what is the core of Scala and what are the dialects and uh I also tried to contribute here in the talk just to identify to nail down what the core of Scala is now that doesn't mean that you it must be your choice to say it's the same core of Scala any team could decide to say well we want Scala plus X and uh that's just our version or we want Scala minus 6 that's our version so everybody is free to do that but if you want guidance from from me then that's that in in that talk that's the guidance I gave you so a functional programming course on corsera and shortly afterwards tried to challenge myself uh by uh taking a follow on algorithms and Analysis course and I you know made the resolution that I was going to do every single assignment uh while holding through the through to the core tenant of immutability and uh you know right everything using Scola for that assignment for every single algorithm and I realized that you know the years of training that I've had uh with all the algorithms that have been taught to me as uh you know being imperative where you know if you're trying to find the minimum cut of a graph with a million notes then it's pretty hard to do that without giving up giving up some of those core tenets that you learn uh when you first learned functional programming so I was wondering if you have any insights to share on is there a class of problems um that functional programming in is inherently not good for uh what is that class of problems and what is functional programming good for okay so the question was uh so the situation was that uh you um took a class in algorithms and wanted to write all the algorithms immutable in an immutable fashion and found out that sometimes it's very difficult to do so uh for instance to do a minimum cut of a graph of a million nodes and question is is functional programming not suitable for certain kinds of applications so the first thing to say we actually have a a paper in Communications of the ACM uh which is about Scala and the examples are all about graphs it's not minimum cut cut cut it's topological sort but still it should give you some inspiration how to go about that so if you look at that then what you will see is that actually we don't shy away from using mutable State and in fact Scala is not about shying away from mutable State it's about encapsulating the state so what I mean by encapsulation is not the objectoriented form where I say well I give you a getter and Setter instead of a VAR then you still have the VAR right that's not it's really to have state that is not visible for the application that's just there as a way to be more more performant so for instance your minimum cut if your minimum cut did a did an imperative traversal of the graph but it wouldn't modify the graph and the the result is just essentially a list of nodes that's a minimum cut I don't care whether you use mutable State internally or you don't what I do care about is that as systems grow in your interfaces of the components you should minimize mutable State because we know that's a liability that's antim modular that's very difficult to deal with so that in the examples I gave you with the compiler where in the same main I'm actually quite proud of the mutable State I use in the compiler and how clever it gets hidden so that it just in performance and uh essentially I don't I don't pollute my interfaces with mutable state that for me is an achievement if you can can do that now to get there it's true that if you come from imperative programming to get there it's actually good to have sort of a a period of of of discipline and fasten thing and say well I want to really force myself to be purely functional for a while because only afterwards maybe then I can Feast on meat again and drink alcohol or whatever so so so and and that's that's a good that's a good school that's good discipline but I would say well and it in definitely it's also very interesting to see what can we do in the implementation of these functional algorithms that they remain uh that that the implementation is purely functional but it's not an end in itself so to to say well sometimes we have to admit it's harder or it's less efficient to do it purely functional than imperative and that's also okay can we nail it down what's really harder purely uh in in in functional than imperative that's difficult because essentially I would have said a while ago well anything with essentially graph problems but actually the functional guys keep inventing new new uh abstractions that turn out to be extremely hand handy handy so for the graph problems for instance there's this concept of lenses that actually lets you mutate in quotes a graph in a purely functional way and that's essentially a library of lenses but you have to sort of learn that and know about that it's a it's it's another concept so I don't think I want to make it tie it down to some set of things where I say well no this imperative will always be better for that it's a moving Target more question uh maybe one from the back y so word on the street is that you're not necessarily a fan of the idea of s of Hing Scala with things like Scala Z so I just want to know your thoughts on libr like that shap um hascal ising Scala with Scala Z and shapeless um yeah well for me that's is essentially that's a dialect so I personally haven't found occasion to use it a lot for the stuff I do uh but uh that might be just the stuff I do so I can't really talk about uh and I've I've there are many people I respect a lot that use it to great effect so I I can only say well some people use galaz Z and shapeless and do great stuff with it and other people uh haven't found a lot of usage for that and I think we what what we would be good to make have maybe at some point in analysis to say where does it help and where do do we find not a lot of usage for it uh but uh it's difficult to say really right now that's that's all I can say last question yes I The Reel and the worksheets was really helpful in learning the material and learning schol um what can you say about the future ability of language the intruction in the going to improveing the debug a little bit challenging right now what what debugger are you using for Scara the one from eclipse eclipse the eclipse debugger so debuggability actually um I'm not exactly sure what version that is uh but we have have have been working on the debugger and I think think in particular the stability has improved a lot so maybe maybe that's already addressed um I'm not sure whether lambdas will help because in the end they essentially the code with lambdas Will in the end be very very close to the code that we have now only will be generated at load time rather than at compile time but that shouldn't affect the debugger um one thing that I personally find quite exciting that we're currently uh looking into is uh asynchronous debugging so that means that if you have a reactive system with threads uh with with uh with with events then typically your call Stacks give you very little information they tell you almost nothing so what you really need want to know that if you have something happening well it got prompted by a message so who sent that message and what was was the situation at the caller and at the caller of that caller but then you sort of hop between threads so it would be very nice to have that as a debug View and we're currently looking into what we can do there good thank you all for the [Applause] questions