Scala By The Bay 2019: Win Wang, Rsc: Scala Outlining for Distributed Compilation

Scala By The Bay 2019: Win Wang, Rsc: Scala Outlining for Distributed Compilation

Recording: Scala By The Bay 2019: Win Wang, Rsc: Scala Outlining for Distributed Compilation

[Music] thank you very much for coming to my talk my name is win and I'm at Twitter I'll be talking about RC Scala outlining and how they all plays into a distributed compilation for hopefully faster Scala compiles so first about me Who am I why am i giving this talk so I want a language we'll see Matt Twitter and yesterday my fellow team member Olaf creator of medals gave a talk about medals so I guess we're kind of double dipping but it's fine because his talk was awesome please check it out it's about how Scala IDs have evolved and will continue to evolve in the future so previously we were the advanced Scala tools team and then we expanded our territory to other languages because we decided that oh hey we could just support the entire company but in verso so we're still a tooling team it's just that we're we're focused on all the developers and speaking of tolling I have inadvertently worked on tooling at my previous projects and my previous companies I just didn't call it tolling I just did not know there was his word tolling well I didn't know but I thought like this boring word that was like carpentry or something I don't know but it turns out stuff like compilers editors all part of that were tooling because developers always have to work with these tools so in terms of compile times I have complained about them for several years so this started when I was working in Java and the Java compile time was four minutes on my dual-core MacBook and then I begged my managers to give me a quad-core MacBook which promptly brought the times to two and a half minutes so I was like yes I can actually like save some time when I changed my branches because I had like five feature branches open at any given time and as for ci science I was like man 20 25 minutes to like do do a quick over on a PR never and then I started Scala and realized that that was amazing talking about 20 minutes Wow right now I'm at a now and my clean compiled times are like 11 minutes yeah so that now it's not great for the Scala story but I don't think I really need to motivate compile times anymore like I think everyone can agree that faster compile times is generally better for developer experience so how we're gonna actually attack this problem of faster compile times it's going to be a little different than how he's talked yesterday from data Brooks it's going to focus on outlining a lot more than remote execution and other possible technologies so the agenda is going to be first introduction to and the history behind RSC and outlining its current status in the eCos skull ecosystem and then challenges we face in production izing separate outliner and finally can you use outlining today I think that's pretty important for for us to say hey listen this thing actually provides any value at all so let's jump into it so who knows what outlining is or has heard of outlining just raise your hands all right just a select few of you what about RSC have you guys heard of RC surprisingly more of you huh that's really surprising and what about C++ headers it's heard of that all right now who wishes they never heard of that guys oh yeah there you go all right nice so we'll be talking about this very shortly so what is RC RC is an experimental Scala compiler for the goal of faster compilation speed and I really want to stress the two things up here that are in bold so experimental and compilation speed so it's not a fork of the Scala compiler but rather a rewrite of it from first principles with a different goal in mind so the goal in mind is just compilation speed and how we're going to actually do it well at first RSC was actually a type checker so why was it a type checker the research goal of RC is five to ten X compilations for you so you know like 500 percent to 1000% sweet ups these are huge huge numbers why do we even think that we could get close to this so Gregor is Cal sakowski sorry if I butchered the name did a little experiment on Scala type-checking a few years ago I called it kentucky mule and it was to explore the limits of the type-checking speed and he was able to get purported numbers of 40 times faster than the official Scala compiler just for type checking of course this was a very limited type checker but it still was like a monumental like 40 X Wow almost two orders of magnitude right and part of his testing he looked at Java C benchmarks versus Scala C for very similar semantic programs and he found that Java C was six times faster to compile even the simplest of programs which is like a scathing review of scholars compile times so this is this provides a pretty good upper bound of what we can might expect it's a little disingenuous because Java itself was designed for fast compiles but it I would say it serves as a good motivation for what we may wish to do so I said RC started out as a type checker but then what happened if you look in the RSC repo now it doesn't really it doesn't have a type checker so what the way that this kind of evolved was Yui we actually had to produce value in a company so Wow never heard of that before I'm joking so Twitter stew hood who is the lead of the bill team here at Twitter he looked into how C++ builds how large C++ bills are done and realized that it was highly parallelizable because C++ files only depend on the header files for compilation so bright idea what if we produce C++ header files except we call them Scala header files and in fact instead of Scala header files we call them outlines so that's how outlining came to be so instead of a fast type checker we decided to evolve our se into a fast outliner okay so what is what do these outlines even look like well you have normal Scala code hopefully normal on the slide and this is what the outlines look like so we have these public type signatures and a bunch of class definitions and scopes and on the lower right you see that we don't look at invisible members so some for example private deaths and we gloss over a lot of details like the implementation details of the right-hand sides of these definitions so we we can save a lot of work and we can pretty much get type checking and thus type inference and cogent and pretty much most of the compiler so yeah that's a one real trick where you just like delete most of a compiler and you make it faster it's pretty cool but okay so we delete most of the compiler how do we actually make use of this thing so this is a hypothetical Scylla project let's just say has like four targets so we're gonna we're gonna illustrate how outlining plays into faster compiles so a target you can think of it as a collection of files that may have crazy insert dependencies so they get compiled in the same compiler invocation and targets can depend on each other and in a monomial you have lots of these targets mmm sorry and these dependencies mean that they cannot start compilation until they're well their dependencies finish compiled so in our illustration pnc depend on a so a has to compile first B and C will continue compiling afterwards in parallel and then D will start compiling out there C is finished so this is a normal Scala compile as to say it's fixed like four point five seconds or something and if we have outlining then what we would do is actually we reduced the time to compile by 50% well reduce it by 33% outline this is under the assumption that outlining is twice as fast as a typical Scala compiled so we actually outline and compile in parallel but notice that B and C do not depend on a is compiles anymore but rather the outline of a so it can start earlier and in the previous graph which we have a two cores available parallel them so we have two tasks running at any given time whereas in this graph we see that there's actually four tests running at any given time now that's not the full story either because yes we have four tasks out running at any given time but it's only really the middle parts that are so that middle one second of time that our four cores may be fully saturated but what we care about is the effective parallelism so in fact if outlining was even faster if it was let's say instead of twice as fast it was ten times as fast then we would have more effective parallelism and we get down from 4.5 seconds to two seconds and interesting thing to know is that D which started out finishing last because it was all the way at the end of the depends of graph now finishes even before a we start a compilation first so just a interesting to know about a interesting thing to know about the property of outlining when it's when it gets as fast as ten times faster and in fact we can go even further than this so it turns out that what if you just make a compile of a target depend on the outline of itself that's kind of interesting so that means that we can actually split up the target as much as we want so in this case we split up the target into two parts and call the sub target parallelism because now we're paralyzing within the target as well and you see that this gets us down even further despite the subs are the outlining overhead being now 20% so it's only five times faster and in a parallel world you will find that contention will actually slow down your eyeliner by a significant amount because it's so fast already so this 10% to 20% number may seem like okay where we may be just playing in random numbers but actually you can see some sort of effect like this in real life of course it won't be like twice as slow unless something like that and in theory since outlining is the serial portion of a compiled you can think of it like that then you can paralyze these sub targets into as many little parts as you want except you can't because there's overhead to actually starting these compels but in theory you can so how does this actually get us faster compiles right like we in this chart we have eight cores worth of parallelism and thankfully now MacBooks can have up to 8 cores but in the past it wasn't always the case and what and this was a very simple project right like it just had four tart 4 targets what if we had a huge project how will we have the core count to actually deal with this so that's where distributed compilations come in and stew gave a talk at Scala days back in June about this it really focuses on the confluence of three separate technologies so we have RSC outlining to unlock faster compilation well that's cool but how do we actually use it is we need the built tool to change the way that invokes a compiler in order to serve an outlining workflow or a pipeline compile and thirdly we need remote executions to take advantage of the available parallelism now that we now have in the project and really is all these three things that come together to give you faster compile times as how we showed yesterday instead of remote execution you could in theory have a huge box and just paralyze over all the cores but we won't really get into that until maybe if you guys have questions about it later so that's really great why don't we have this yet turns out that when you throw away most of the compiler you get a bunch of drawbacks so these are some of the drawbacks one is right we threw away two type checker we don't have any type checking so no type inference public return types now need to be explicitly ascribed and it's not just public it turns out it's visible so protected and a few more types and also macros so what's a macro story here macros require full by code so we can't depend on outline specifically macro implementations right when you're implementing a macro you actually need the full byte code of your dependency unless you don't depend on anything for the macro which in practice this seems to be more the case but this is something to think about the you need a transitive dependency tree so they could this could break you're outlining workflow cool so that's a bunch of hypotheticals what is the reality of it all right this is Twitter's response to lower compile times what is the community's response right it's not parallel compiles are not just something that Twitter's auto of obviously so there's a bunch of related work as mentioned before a topple list is c plus headers so C++ headers and a way that kind of compiles has been around for a long long time and interestingly Google actually has a header compiler or outliner for Java called turbine it's still experimental there the work on it is still I'm actually not sure how much work is necessary for Google to adopt it but also bezel has IDE hours but in terms of pure Scala work and in the Scala ecosystem there are three major related technologies so one is Hydra and that's a parallel Scala compiler it was built by Julian Drago's and the miracle daughter from epfl sorry if name pronunciation is bad and the idea is it just works it just works as a drop-in replacement for the Scala compiler and then we have blue which is a build server and it was mentioned in Olaf saga actually so the same thing that Olaf was talking about with integration to metals and vs code and better IDs bloop is actually a pioneer of build pipelining in Scala so one of the first prototypes that actually made this type of workflow it was a technical proof of this type of workflow and in contrast to our outlining techniques oops pipelining is all in memory data so here we can make it work with remote executions it's just less there's more work to be done in that area outlines by contrast are actual files that are produced by an outliner and as of Scala - 12:9 Scala C can actually now function as an outliner and this is all due to the work of Jason Zog at retronym and this is really great stuff because it means that outlining is now part of the official compiler we don't need any spoke technologies to experiment with it although it is you know behind a - why experimental option but it has great properties so the generator outlines from this conform to Scala meaning that they are correct and you can use them for later compiled so remember the outlining graph you can just it just works so does that mean that RSC outlines are not correct yes it does but it's not correct in a in a few buggy ways but it does not mean that we do not have correctness in our C as an example we have compiled at three hundred thousand lines of Scala project with RC with no bike-o dares so back to why outline though since type checking is only skipped for definitions with type descriptions so the way that this is influenced it is it just skips type checking if it sees a type that is explicit and what that means is type inference is still there it's just that if you add the type it will be a bit faster we still need to worry about macro expansions and actually the outlines that produces aren't dot class files but rather dot stick files which scalzi understands but other tools will need to have a little additional work to integrate so that's really nice how do we actually ship so this is kind of going to focus on RAC and the challenges we had in in shipping and outliner and ok so these are the three ingredients that we mentioned from stew stock basically you need RAC compatibility so explicit result types you have to deal with macros you need the build tool to understand how to invoke the outliner and compiler in the correct order caches dependencies etc and you need remote execution or a sufficiently large box except you don't only have three things because when shipping you have validation issues and user experience issues some well the users accept this is also still not enough because remember the the goal of all this was performance you're supposed to actually deliver some value to users and this is a this this means that there are six things that we have to kind of think about and this is what we mean by RC compatibility so let's just kind of like go through the things that are necessary on the RSC front or the outlining front so this is an example of a publicly ascribed return type so instead of just foods go to whatever you actually need to have a type it's of type sequence of string and we have a automatic Scala fixed role that can actually ascribe these types for you so you just run Scala fix it's great 99% of issues are pretty much caught there are a couple things with like defined types and laces and so existentialist but most things are they just work out of the box but that's also not the full story right like this this previous thing is okay what is stuff that map whatever oh who knows but it's not so good when you have something like def X my class is called new my class yeah we've basically devolved into Java at this point which is not something we want to tell Scala developers and the second example is this is not in any version of finagle so this full disclaimer this is just an illustration but this is what one of the types in Twitter's finagle library would look like if you ascribe the public return type yeah have fun putting that in your PR and again visible members are actually not all the things that you need is everything would type inference so for example if I have a class with a constructor that takes a T then and I extend that class with a in it then actually it infers that the type of the class is Bar of int so you need to actually ascribe this well the Scala fix rule needs to describe this really so besides RSC compat which is a Scala fixed rule what if we just want to have explicit result types so Scala fix comes with a built-in rule called explicit result types and as of zero nine eight Olaf has made great contributions in the use of this rule so you can and then this is just a scala fix rule that you can use with a lint but you can actually enforce such a rule at compile time so have a compile error with wart removers public inference rule and this may be interesting if you use Scala C - by outline because then you can enforce public return types to prevent any backslide from performance issues if people start forgetting return types everywhere so validation just really quick how do you validate code well you have tests but now we're rolling out an entirely new front-end so how are we gonna make sure that we don't destroy Twitter so one is you can check by code right you run a normal compile and you run your outlining workflow compiled and you just checked the bytecode that's really great except you just built your artifacts twice probably like the thing we don't want to do you can actually also just check your outline so remember I said that the yl line outlines were perfect and you could just use them right off the bat well it turns out that that means you only depend on outlines and if your outlines are correct then you're by code is correct and what about the user experience so as mentioned you can actually make this a compiler so if you outline with rse it actually just admits this compile error says hey note we didn't have a type here here's the line here's the thing you need to change and what removers public inference rule will give you a very similar output and okay so let's get into can we actually use this thing so I talked a lot about oh yeah this is the theory this is the ideas what about the use of this well yes you can use it today the newest pants will support outlining workflows and both RC and also Scala C's - why outline however a Piazza exists everywhere so macros again you have to annotate your targets well that's one way to fix it just annotate which targets must be run with a full compile and also performance like yes you can but why would you unless you have good enough performance numbers that you will be able to convince people Hayes just put all your result types in your code so what are our performance numbers well we have a 25% speed up on 16 cores so these 16 physical or is not hyper-threading and this is on three hundred thousand lines of code for Scala is the actual lines of code in brought to you I think it's four hundred thousand so that includes Java and other languages and this number does 25% speed-up is just very experimental so make of that what you will but this is like a technically proven speed-up and this is without remoting but if we had even more course then perhaps we could do even better so actually what about a more normal target where you don't have to thousand lines code this is an even more experimental number but this is something that I can run on my laptop you know you can shut to users and they won't see a slowdown although this is on eighty-eight thousand lines of code for Scala and the reason you would see this 16% speed-up for a smaller code bases it inherently has less parallelism until you use outlining whereas a three hundred thousand lines of code could base well there's a lot of things that are depending on everything and it has much more parallelism to offer Ralph that this is super experimental numbers but I thought it would be good to share our results so far so performance actually you can talk about other things which are doing performance right we've been talking about compilation performance a lot but what about IDE startup time indexing time cache fetch time for remote caches and it turns out outlining can possibly help in this regard as well because of lower jar sizes so the class files full class files is by code right the factor between that and outline only class files is a for X Factor and for that's why I outline so dot sick files is a 9 X Factor and then so I also added the semantics DB outlines which is something that our SE can produce and in fact our se does this thing where it creates semantic DB information and then turns it into outline information and that's a factor of 31 X what does this actually mean so currently IntelliJ supports class files all right you can just have class files and IntelliJ will index them and create information from them to allow you to use those signatures when you're editing so like when you're complete when you're Auto completing when you're going to definition you can actually use these outlines if your outlines are much smaller than IntelliJ could in theory index faster and also maybe if you need it to fetch from your cache you could download things faster if you have a worse in Wi-Fi connection and the dot stick outlines produced by away by Scala C well IntelliJ doesn't understand them just yet the reason I highlight them in yellow is because there is a there's a ticket on youth track that I IntelliJ may or may not the people have chaplains may or may not wish to fix where they can understand dot sick file so this gives you another fact there are two acts smaller char sizes so in summary that was a lot of info but in summary we want to say hey RC is this real thing and it has given us real results for numbers it's a I think it's a great experimental platform for hacking on performance experimentations outlining is a real thing we have it in the official Scala compiler we have a production nice build tool that has an outlining workflow working and I think we can be quite optimistic about the future of Scala compiles outlining is a great advantage that I think will prove its worth in the in the future and even now we have a technical proof of its advantages and this work has a lot of contributions right it's not just me or my team or even just the author of RC there's a lot of people who have contributed to well Scala because they Martin he created Scala and then a lot of people have a contribute to faster kampala times faster build times it's just a lot of great competent rebukes from everywhere so thank you for coming to my talk I don't know have [Applause] don't know if we have time for questions but okay there's any questions I can I can take them now raise your hand yes me bring the mic over to you sensitive it the generation of outline based on the complexity of the type signatures for example if I added contravariance and covariance and multiple you know functors and the type so I don't think it actually depends on the complexity of type and the reason behind this is there's no type checking so you have to explicitly give give me the type or like give the outliner to type and well it's very simple I just say oh yeah okay I'll just trust you know it's kasi does do that it just skips type checking yes one thing that you should was a you know the the the two-phase compilation depends on building the dag first of the files and you're gonna compile and then you you call the compilation can RSC take a set of files just like as if I wanted to use it as a tool to plug into some developer tool can it spit out that dag for me relatively easily no it's not a build tool so III understand that without what I want is for it I input list of Scala files output dag of Scala files the dependency graph of the Scala files because clearly you have it because you build it okay no honestly cannot output that that's a pretty interesting idea though like we could actually just make that happen if that's useful in some way so a better answer may be RC can output a semantic VB and the outlines and there's a bunch of tools that use semantic DB to build up a possible dag yeah the only problem with that though is you can have dependencies in like private in the implementations of methods so I'm not sure if outlines would be sufficient for your use case any other questions nope perfect great thanks Winn all right thanks everyone [Applause] [Music]