SBTB 2014, Haoyi Li: Cross-platform development with Scala.js
Recording: SBTB 2014, Haoyi Li: Cross-platform development with Scala.js
hello everyone my name is ho and I'm going to be talking about crossplatform development in Scala JS Scala JS is a scalar to JavaScript compiler which lets you take your Scala source code compile it to JavaScript and then run it in the web browser like any other JavaScript so it lets you do cool things like share your source code between client and server or get type checking in your web apps which are traditionally made but with some something like JavaScript or coffee script which are kind of soft squishy languages that keep making mistakes and the code you end up with ends up being about a few times slower than raw JavaScript quite a lot slower than Scala but nevertheless it's still fast enough for many purposes it's a lot faster in Python for what's that what that is worth um so what's wrong with JavaScript and why would you want to use scjs so at work I work on our Coffey script code base and I find that many of the things that people bump into when you're working in the code base are the same no matter how long you've been working in coffee script how much industry experience you have or how smart you are things like why is this variable undefined or I want to refactor this but I'm scared so I don't so it's that kind of very meal lowlevel problems no one is writing big algorithms to do cool things with the web page people are just struggling against a stupid codebase trying to make it not crash when they make their changes and in in reality no one actually knows what's going on some people tell other people what to do but they don't know what's going on either so just to show you some how scolar JS Works here's a short little demo with scolar JS so um I'm going to set up SBT to do live reloading and over here is a small scholar JS application it defines a point class it runs it gets a canvas rendering context from the browser and sets an interval which starts this little Loop each time the interval runs it runs this Loop 10 times and this Loop goes around the canvas putting points and if if you recognize it this algorithm basically gives you this sininsky triangle draw and the actual algorithm is kind of irrelevant here what's interesting is that I can mix things like scalar sequences and scalar points with things like canvas rendering contexts which are defined in JavaScript is this fellow canvas rendering cont text and aren't even defining Scala at all what else is interesting is I hooked it up with the web browser such that when I change stuff in the code I should get X SBT log spam in the browser and it should reload on its own so the first time you change things it's kind of slow after a while it speeds up as the compiler warms up and so this is kind of interesting but let's try making it more interesting so I'm going to try turning this into a video game in the next five minutes so to make it a video game first I don't want to run this 10 times I want to clear the canvas every time I don't need Corners I don't need counts I'm going to rename P2 player and don't need this anymore and I'm going to make the player of 10x 10 rectangle so this is blowing up doesn't compile because of this fellow hasn't been changed yet let's make the player white so as you can see when you screw up in scolar JS you get a nice compil error not in not just in ID but also in the browser in SBT console so has this guy worked yet okay so now it works so I have a player in the top left which is white so let's move him to the middle and let's look at adding some input so the way you work with input in Scala JS is the same you work with input in JavaScript you basically use the Dom apis for example one thing can do is say I want the set the on key down method of the Dom and you can see that I get nice doc strings in the browser it's pretty neat even Handler I I grabbed them from mdn so this guy is a dom. keyboard event keyboard event and I can do things with it so should do that okay so what I can do with this is for example I could print out the events key code let's say that's the integer which represents a keyboard event and I get numers so in order for me to Player move around I'm just going to keep track of what key keys I have pressed and I'm going to which is going to be a set of integers I'm going to just add the keys whenever they get pressed down and I'm going to remove them whenever they get pressed up so that is on key on key up and so here if I print out the set of keys you can see that I have a mutable set that depending what I have inside will change and I can change this to instead of printing it out I'm going to set if the keys for the key code is a constant basically a bunch of constants that I made that holds a bunch of keys so I don't need to remember what number is which plus equals 010 should we probably be negative -1 let's say two for now um down will be positive two and I'm going to make left and I'm going to make right so this is going to be -20 this is going to be positive two 0 and everything worked now I can control the player moving around the screen so that's quite neat it's been let's make it a bit faster so I can so it doesn't wait so long so it's been about 3 minutes since I started now kind of working video game so in order for video game to be interesting you have to be able to die so let's give myself some enemies so enemies are going to be a sequence of points um let's fill it so I don't need to make them manually I want 20 of them and each one's going to be a random number random point from 0 to 255 um to 0255 I think I did that right let's move this down so it's fits within the screen and for each enemy for enemy in enemies I basically do the same thing I said I'm going to make the enemies red so let's take this guy move him down here and make him red and now I have okay that's wrong because I'm printing on top of my player so I should be doing and printing the enemies X and Y so I have a bunch of enemies not very interesting I can't die yet but if I add a little if statement that says if I'm too close to enemy so let's say playerx xus enemy dox less than 10 and player doyus enemy doy less than 10 um so I I probably need a alive flag somewhere so I'm going to set that to start off true and this probably needs to have a math.abs because otherwise I it only detect One Direction not the other so math. ABS so I shall move this down so it fits alive equals to false and if I'm alive I do this stuff I don't move if if I'm dead I don't move and my context. fill style which is a JavaScript API I'm just using it's going to be let's say gray to represent dead um let's get R of the space so if you let that spin up a bit you can see it's getting faster to recompile now if I hit something I stop moving and I and I'm I turn gray the last thing to do in this game is to make enemies move and that can be done pretty easily just by saying enemies equals enemies do map e goes to a point um e dox e doy + 2 and now you'll see all the enemies are moving and they all run off screen and disappear so I probably need to make it E y+ 2 moduo like the size of the screen and then they won't disappear so now they're moving if they hit me I die um that's pretty neat I have a kind of working video game if I want to make it harder I can just make the enemies move faster so let's make them move it double my speed and here we go and it's it's pretty hard okay so what's interesting here I've made a working video game in the last five minutes so let's see look take a look at this game currently it compiles down to JavaScript and if you look at this thing if you'll let me see because it's kind of small let's make it bigger the JavaScript bundle compiles to is 764 kilobytes so this is without any fancy optimizations it does some actually does it's with some fancy optimizations but without all the fancy optimizations so to get ask it to do more fancy optimizations I swap out fast optimized JS with full optimized JS this normally takes another maybe 10 seconds per per run which is why I don't do it all the time because it takes a while but it's not so long that you can't do it once before you actually put the code out on the web for for example so has it already optimized it I think it has okay so that's done so I'll change it to the optimize page and you see looks exactly the same except now the final thing is 126 kilobytes of JavaScript without gzip so that's pretty cool um let's spend a moment to reflect on what we have done so if I open up the code that I just compiled which lives in here the fast optimized code I hope it doesn't crash my editor looks like that so it's kind of verbose and long what the fast optimization does isn't compression as much as dead code elimination and to some extent um inlining so it makes the performance better it makes it go from 20 megabytes to less than one megabyte and but still it's kind of readable so if you look at it and kind of figure out what's going on so if I look at for example collection mutable I miss spell mutable C SE where is seek maybe don't have seek oh because I didn't use mutable seek but if I use mutable to got eliminated I use mutable set no okay H I don't know what happened to it but anyway so this is generally what the code looks like if you look for my code which lives in um scal JS example Scala JS example it's basically here so the code kind of a Bast but it more or less flows the same way because Scala is actually not that big a difference from JavaScript in that you call methods the same way you pass arguments the same way get return types the same way except it's really veros on the other hand if you look at the fully optimized version it kind of looks like this so this is what the Google closure compiler does your code it makes a lot smaller you can't read it probably but in because Google conos closure compil is correct because lots of smart people work on it it doesn't break the code and the code works the same way like here so that's enough of this this demo and let's go look at what something else so so on what some guys you may be thinking may be what can I do and what can't I do in scolar Gs I've show you can use collections I've show you you can use a Dom but what else is available or not available so you can use all almost all the Scala standard Library you can use some small subset of the Java standard library but not most of it you're not going to be use be able to use or. oh my god. corba for example um you can use macros you cannot use reflection the reason you cannot use reflection is because if you use reflection you can't do dead code elimination anymore and maybe we could work around it by with fancy annotations like progard does but we didn't So currently just no reflection and you can use all of the pure Scala ecosystem so if your program is in Scala and does not depend on Java specifically or Java Java virtual machine you can use it someone ported Scala Z someone ported shapeless in case there any crazy super functional programming people here and you can't use things like scal test unfortunately because it uses a lot of java apis and uses a lot of reflection which is why I wrote my own microtest Library which is kind of similar but without all the reflection and Java stuff other things you cannot use jvm operating system networking you can use browser networking so you can use a new hotness like websockets and Ajax calls um you can't use openg you can use webgl can't use Java libraries you can use JavaScript libraries including quite quite big ones I'm going to show you a demo using chipmunk JS later which is a several thousand line of code JavaScript physics engine and you can use all your old Scala like tooling so things like SVT incremental compilation works great intell doesn't even know it's a Scala JS project it just works and you can use a chrome console for debugging so you can see I have things here and if I do something bad like let's say I go here and make it blow up um throw new exception boom let's move this down a bit just to see what's happening so silly me I okay let's make it do the fast optimization so it doesn't take so long so if we spin that up spin spin spin up SBT if you spin that up you'll see where when it blows up you can nice stack Trace in the browser just like you would in the jvm except inste being in the console you get in the Chrome console so if we run this whoops I'm still running the optimized version sorry Dev Dev so you get the exception and and you get a bunch of weird looking methods with Scala stack tracers so that's pretty interesting because we have Source map so it's kind of like coffee script we just so Sebastian set that up and it works great and for example you can see this blew up on line 62 I don't actually know if that's accurate yep there's accurate Okay cool so so now for a bit more a bit of show and tell so I've showed you a small thing you can make in 5 minutes so here's a few bigger things that took a bit longer so first is this little to-do MVC application which I made a while ago just to show off Scala JS you guys may have seen this kind of application before because people use it to show off like look how cool my new JavaScript framework is I can do this in less code than the other people can so it basically works the same way as any of the other examples if you Google Tod do MVC application there'll be a whole bunch of examples and there's even some to do MVC application company which kind of owns the copyright for this thing um but what's interesting is if you look at the source code for this guy it is 150 lines of code and that involves all of the HTML all of the which is kind of lives down here all of the event handling all the updating and all of the state that lives on the in the JavaScript so here's my list of tasks that gets updated when it when I click things if your you notice this little framework thing which is seems a bit suspicious but it's actually like three functions function function okay four functions and basically this is about the third the size of the of most other JavaScript implementations of the same application so I think reactjs is about 400 something lines of code backbone JS is maybe 800 um there's an angular implementation which is about 300 I think that's a closest to what we have here that's including both the HTML of angular Z HTML and the JavaScript excluding CSS because we all have the same CSS so that's pretty neat and it's kind of a different thing from what I was showing you earlier where it was just a canvas drawing stuff on here you actually access the Dom a lot you're rendering big blobs of HTML here using my Scala tag HTML Library which takes a bit of getting used to but it's very concise to use um and generally kind of just shows off that you can make websites with this kind of like it works I haven't made a a very big website with it but in theory it should it should work fine next thing to show you is rooll so rooll is a cute little application that is what was what first got me into scolars and what it basically is it is it is a 2d platformer that is written entirely in Scala JS so if you're at SF Scala a long time ago I showed this soft before when it was still black and white and didn't really have many levels and now I have a whole bunch of levels including pretty fun ones like this and the goal is to roll your ball into the center of the screen and the gameplay logic is all written in Scala it runs in the browser if you look at how big it is how big is it reload this is about 280 lines kilobytes of SC of JavaScript gzip to 80 which is reasonably small about the same size as jQuery so that includes the whole scy standard library that I'm using as well as maybe 2,000 lines of Scala code so it's actually pretty reasonable and I have a bunch of levels some of them are pretty high like this one because if you touch any of the red things you die but in general this kind of just shows off like you can make pretty complex things with Scala JS maybe I haven't tried making big websites yet but big 2D applications work uh great so the last thing to show off is the fiddle which is a little website I made which if my internet doesn't die on me we let me run Scala JS Snippets in the browser so here here has a little one which makes a little oscilloscope is kind of a cute example because it's pretty small sequence of a um sequence of V of functions that you can call like literally math functions and it just plots them out over and over and you can change things and it rerenders you can change things to like purple I want it purple and I want this to be twice as fast because I'm OCD so you look at it it goes twice as fast nope I screwed up my math that t as fast okay so and it lets you draw things and otherwise do things in the browser without having to open up uh open up your in your editor what's interesting is this whole website is written in Scala and so is a back end and so is a front end and all the ajaxy stuff is done in Scala and in total it is 1,000 lines of code for both front end backend all the Ajax all the compiler interfaces and you can do interesting things with this 1,000 lines of code like you can do anything you want for example here's a two dnbc application I basically copy and pasted the whole Source in here and it works which is quite neat and here is array Tracer which is a my favorite example because it's very pretty and it's written in Scala here has a scene definition and it just kind of shows like yeah this actually does work it's not just one plus one anymore and it's not just like oh look I put the I turned the Dom red wow that you actually can do pretty neat things and it's not really it's not super fast but like rate tracing is never going to be fast so that's okay so that's it for the Show and Tell I hope you guys found it interesting let's move on to something else um so I promise so some things that why you want to use scalar JS is that scolar so you may want to use scalar JS instead of SC JavaScript because scala's nice you may want to use it instead of something like cotlin or Dart because we have a lot biger ecosystem than they do most of it for free like the dart people have a whole team working on their Dart Eclipse plugin and we have two idees that we got completely for free because they don't even know they're working with scyla JS um you may want to use instead of Swing apps because sending emailing people jars and getting them to run jars getting a class path right is actually pretty difficult and I've tried it before and lastly because front end development in JavaScript is actually pretty fun and it can make a lot of progress sorry front end development in Scala is pretty fun it can make a lot of progress very quickly because of all the tooling that you get so that's for the the short examples let's go into a slightly longer example so this is going to be a server client application and I'm going to cheat and use a reset hard head I'm going to use a uh prepopulated example because it's a lot of code to write out you can see just setting a servant client has a few dependencies and you get spray I need to get AKA I need to get a bunch of things so rather than typing it all out in front of you here it is so what we have here is a project with three main modules when you reload you'll see has client it has server and it has shared so client is basically a Scala JS application which renders an input box renders some text not some text some HTML and on click it updates the text has rendered server is basically a spray can application which contains a little HTML frag fragment up here which is a skeleton of the page as well as a small spray route which says for the root just give that skeleton otherwise give the resource and for any API calls wire it to the right place so this results in a small application that looks like this in total is about 100 lines of code 60 here 50 there and basically you can like look at your file system it's not really very useful maybe you really want to look at all the things that your comp compiler spits out you can but the point is just to show a mini example of how you can use scolar as a client and server so what's interesting about this application is that you can do things like jump you can do things like share code between client and server so for example um the simplest code you can share is probably an interface and over here you'll notice that if you look in the client where is the Ajax call well Ajax calls obviously here but where are all the strings that are normally associated with Ajax calls well I don't have any strings it is a strongly typed method to an interface and this guy will get pickled and sent he's a string he get pickled and sent automatically this guy who happens to be a sequence of strings will get pickled on the server sent back to me and unpickled automatically and this fellow has a pretty readable readable signature basically it you can call any Ajax function that returns a r and it'll give you a future of R which is kind of like what Ajax is right it's a asynchronous request which may fail and there is some macro magic which I've done in order to do things that capture these guys and pickle them properly because that doesn't happen by default because not really part of the Scala type system but anyway the point of this is that I have a Ajax call here if I want to find where it is on the server I just click this thing and click this thing and here I am so I don't think in my experience working on the web this has never been the case it usually involves lots of grapping and some swearing and here your editor can tell me oh here that's what Ajax call is and here's who's using it and here's where it's defined and if you want to do things for example let's say I want to go to my server which is this guy and I want to make it return not just the file name but also the file name and with its path so map F goes to F I think it's prefix plus F this is probably wrong in some way but it'll serve to illustrate um you'll see that it's compiling and then it yells at me saying nope you did not change the signature and I did not change the signature up here either and and now it's going to tell me I didn't change the signature down there because I didn't name path let's make the name bold name plus let's do that path okay and now you's see after everything is finally compiled everything works again which is I don't know how many of you do web stuff but this is very really the case when you're working with coffeee scripts because when when when you're working with coffee scripts you have nothing to check these things for you and you end up with a lot of grapping maybe a few crashes in production that will then get the sales team page which will then page you which should then result in emergency push the pro to fix some typo or some call that you missed but with scolar JS it's tied together very nicely such that if you make any sort of mistake in the client server interaction the compiler will tell you whoops you forgot something and like even before you compile it if I make a typo for example I fat finger it it tells me right in edor no there's no such thing so one more thing that you can do is that if you want to make If instead of returning a tle maybe one of those guys who hates tles and you want to return the case class you can do so so I'm defining a case class which is called file data um let's give it a name and a path and I return a file data so you guys probably already can guess but this guy's going to keep yelling at me until I fix all the call sites so I probably don't even need this thing anymore because it's of type inference I didn't need him in the first place and so I need to fix him and I need to go to the client who is no longer at getting tles and returning a file data and everyone knows it's returning a file data even my editor knows it's returning a file data now and if I get it to reload you'll see that again it works which is pretty nice and one other interesting thing is if you look at the network console to see what is actually happening let's say I do a bunch of typing here and this is clearly wrong but ignore it you'll see that I'm actually sending down nicely structured um nicely structured Json well for one I'm sending up nicely structured Json if you can see and I'm receiving back down NC structure Json which is all generated automatically by the by the macro well not by the macro by the implicits which are in scope if you want to you can change it but by default case classes resolve to JavaScript objects um if you go back to our previous example which I'm probably not going to because it'll take a while tles will just become like a list of two things and a string is just a string so all this is handled automatically for you and if you want to you can customize it most by default you probably shouldn't need to and if you realize what we've done here we have defined a shared data type which we can share client and server and not only can we share this client and server we can easy just like Chuck the data type across the client server boundary and have the other guy pick it up correctly and it's even kind of type save such that if I do something wrong let's say I let's say I make this guy r turn uh java. i. file let's try make this guy um and and you make this guy return a Java i
file well let's make it this guy let's just make him blow up because I it it shouldn't compile if I did this correctly you'll see that oh this the Pickler does not how does not know how to write Java IO files so this is actually quite nice because it means that if even if you try to screw up you probably won't unless you intentionally write a malicious Pickler that will cause it to explode by default the only things you can pickle are things in a standard Library which I've defined for you there's a big list of them so you look at um if you look at this guy wow there a lot of generated files and in place it's you know there's a list of all the standard Library things you can pickle and case classes which are pickled by a macro and that's it so you don't need to worry about things like oh someone's going to inject something which will cause a Constructor to run which will blow up my program and control you you really only pickle and unpickle things that you know statically what they will pickle to or what they'll unpickle from and if just for convenience I provided all these things but there's no there's no magic in this pickling it's actually pretty straightforward and the library which does a pickling is less than a thousand lines of code so this is pretty nice so if we go back to slides a second what have we what have we seen here so one thing that's nice is I can use the strength of each platform such that on the if you look here on the server I'm using spray that's pretty cool on the client I'm using the dom dom that's also pretty cool if I try to use spray on the server it does not work is it IO spray NOP it's just spray just because the module does not depend on spray so it knows not not to let you use spray and similarly if I if I try to use a Dom on the client again you just can't use it because it's not available so that's pretty nice in that you're able to use the strengths of each thing yeah five minutes more okay you're able to use the strength of each thing but you won't accidentally use the the other thing because not only will the compiler yell at you even your editor will yell at you saying that nope this is not allowed and sharing code and libraries and data structures between the client and server is pretty nice so you guys already saw that I have a little data structure here that is shared what you guys saw but did not realize is I have multiple libraries which are shared this Scala tags library is a HTML generation Library which I made which works on both client and server like there's no reason why HTML generation Library shouldn't it's just splicing strings after all and so it does up Pickle which is a uh Pi pickling and unpickling Library I made also works on both client and server and it's also shared and if you look at my SBT project which is here you'll see that I have dependencies on both the jav JavaScript version represented by three percentages and the Scala version represented by two percentages I think I think this guy gets pulled in automatically so I probably don't need him but never mind and similarly for the Scala tags and Scala tags well that's not true because here I'm using three percentage one anyway and that's okay well it doesn't blow up because all it means is you have some extra JavaScript files in the jar artifact that it doesn't actually do any harm so what's back to the slides static typing keeps things straight and keeps you saying as I showed you earlier we actually have three modules and you you can use different subsets of of your apis in each module and because of static typing you're quite sure you won't screw it up so cool demos I've already showed you sharing libraries between client and server what else can you do you can also share you can also automatically rename Ajax calls so in order to do that first I'm going to fix this guy to not return Java IO file and I'm going to fix this guy not to blow up and then I'm going to come here well come on reload I'm going to come here I'm going to say I want to rename this guy the base member to list new name laws and then once it's done compiling oh no the type Checker blew up just now that's not good come on once it's done compiling you'll see that it should well even if for it's done compiling you'll see that over here it's renamed this guy correctly over here it's renamed this guy correctly and it's done compiling you'll see that the actual route has been changed such that now instead of hitting the slash list and point I'm hitting slash list new name new name lws and points and that's actually again a thing that's pretty nice because my experience working with JavaScript and coffee script is that when you do these things there's a like a tinge of fear that goes through your body I don't know whether you'll break things I don't know who's going to yell at you but with scholar JS because the editor can help you do all these things and compiler can check it there is no fear I just named it and it's a new it's a new Ajax endpoint and maybe new name all make sense in your company's business context and so you get find usages that's pretty neat like you probably already know because intj doesn't even know it's a scholar JS project I can ask for it ask it where it's being used you get lots of safety I've already showed you you can't misuse things and you get tool ability so I already show you tool ability around Ajax let's spend a moment to talk about tool ability on the client so you guys already saw me do stuffff on the client but the fact they can have autocomplete with documentation with like jump to definition in case you want to see what this thing is so these are all question mark question mark question mark because they're just stubs for the JavaScript inter interoperability so they actually call the JavaScript attributes and I can say things like oh I want to get all the children which children or I can this children or that children maybe I want this children for child in children child what I want to do with a child maybe I want to get his client height maybe or maybe want to do something else want to get set his Al HTML what does Al html mean oh Al HTML means this which is probably screwed up Contex HTML but if you look at out here you can see out HTML does a particular thing I can if I to see what in HTML does in comparison I can and all this is done without leaving my editor because you have all this documentation on the Milla developer Network because all of it's done in these plain text HTML inconsistent HTML formats there's no way for you to parse it and actually get useful in editor support with it but because I have done the tedious work of going to the Mozilla website and copy and pasting everything with proper attribution of course you now that even when we are in our editor we know that oh this guy returns a string and because of all the stubs I can see that oh for some if for some reason I need access key also returns a string so to wrap up crossplatform development scals is actually pretty fun like you can do a lot of things and as son who works with coffee script and python Network I think that this in theory solves many of the problems I face dayto day like what the hell's going on why can't I do this why is it breaking and if you want to come hang out we have a website you have a Google group where people are doing interesting things the examples I showed you are here on my on the GitHub different branches and um I'm just going to put up a picture of the ray Tracer because it's pretty while I take questions [Applause]