Scala.js: confessions of a backend engin...

SBTB 2015: Julie Pitt, Scala.js: Confessions of a Backend Engineer

Recording: SBTB 2015: Julie Pitt, Scala.js: Confessions of a Backend Engineer

all right so now that I've been introduced let me introduce myself so my first exposure to computers was through the family Commodore 64 um and I I knew the word computer at that point but I didn't as far as I knew it was a game console um I should have known that I was a nerd because my favorite game was probably the only game that's more boring than real life does anybody remember the game lemonade stand anybody no okay like five people um so it's this game where you're this entrepreneur running a lemonade stand and the exciting conclusion to the game is that you don't go bankrupt so I was looking for some adventure and excitement one afternoon and I came across this haunted house game that had a state-of-the-art UI so I load up the game and I see a screen that looks something like this so first thing I do is I type North because I want to go north and that's when I know something is wrong because the computer says North I don't understand try something else so I think well maybe I'll try all caps and then the computer's all stop sh shouting at me I don't understand North so I'm not deterred at this point and I try various different combinations I try all the cardinal directions I try abbreviating I try geoc coordinates and nothing works so after hours I finally figure out that I should have typed go north so I learned two things from this experience one is I really should invent this thing called Google and I make a lot of money um but you know the other thing I learned is debugging by Brute Force sucks so I'd like to say at this point that I then became a mad integer basic wizard to and ruled the world um but that's not what happened because I didn't know that programming was a thing I didn't find that out until I was in college I went to UC Davis and I was a math major for the most practical of reasons namely that I was good at it and I had a lot of credits in math um so fortunately I took an elective course in computer science uh and that's when that moment happened in the basement of the computer science department I ran my very first C program and those two lifechanging words appeared on the screen yeah you know what those words are hello world so my name is Julie pit and I'm going to be sharing with you my experiences using scala.js and I'm going to share it in the form of a story this is the story of how I came to schola JS and really my hope is that you will get a feel for how to shift mindsets from developing a schola application for the jvm to developing a Scola application for the JavaScript runtime and along the way if any of you are new to schola JS I hope that you might be able to avoid some of the pitfalls that I fell into so chapter one frustrations in JavaScript my career path has very much been as a backend engineer building distributed systems fault terer and apis Etc so when it comes to any problem involving you know anything visual or front end my response is yeah I'll let the UI team handle that one and that's worked out fairly well until last year when I became co-founder of a very small startup uh we were building artificial intelligence algorithms and we came to a point where we realized we need to visualize what these algorithms are doing so my first reaction was great I'll get the UI team on that one oh I'm the UI team so I break out my very old creaky JavaScript and I put something together that that is this dashboard application that visualizes stuff and to say that I wrote spaghetti code would probably be an understatement um after this experience I was the spaghetti um so I I do have a confession to make um I made some very rookie mistakes for example I left the return keyword off of my functions my JavaScript functions you know why would you ever need that uh I did that repeatedly and so I found myself back to my old ways of debugging by brute force and it didn't feel too good so fortunately a few months later I see an announcement hackathon for scala.js and at this point I had no idea what schola JS was but it had ajs at the end so I just signed up so I quickly found out what it was so scala.js is essentially a plugin for the Scola comp filer that takes your schola source code and produces JavaScript that can run in any JavaScript runtime so how many of you guys have played with Scola JS already a lot okay how many of you have developed a serious application in schola JS not as many okay so let me give you an overview of the main features um You can call any JavaScript code from scholar code Cod which is amazing because all the JavaScript libraries oh how many of you guys have done JavaScript applications I should ask a lot okay so if you've used uh any JavaScript library you can use it from schola JS the reverse is true you can call any scholar code from scholar JS as well and that helps you define a very specific entry point into your scholar JS application and then this now opens up a whole opportunity to share your code between the jvm and JavaScript runtime which means if your server is jvm your client is Javascript you can now share a lot of code which is awesome and for me coming from a scholar perspective the fact that schola semantics are very Faithfully supported in almost all cases just a little asterisk for a couple exceptions um it's a familiar environment collections work as you would expect them to Futures work as You' expect them to so um that was great and there's a big Community it's growing so every day I'm seeing a new library that's being ported over that's a standard it's a scholar library that now works in scholar JS so I get to this hackathon and the first thing I want to do is to create my own hello world application I just want something that builds basically how do I do that well the first thing I wanted to do was use the tool that I was familiar with which was Gradle um and I quickly found out that Gradle support isn't quite there yet and I was already learning a new technology stack so I decided I'm going to just go with SBT which by the way I had never actually used that's another confession um so there's a fantastic SBT plugin that works really well with schola JS and here's how it works you have your Scala application code and you have your Scala libraries and the spt plug-in and schols compiler will take the all of those and produce a single Javascript file so in this case it's called my my app fastop doj so um there's a lot that's going into this and I recommend going to tobias's talk tomorrow if you want to understand all the optimizations that are happening between the Scola code and the JS but for me as a user I just go cool there's a there's a file here and I'm going to include it in my HTML and I'm done right well not really because if you're running a JavaScript application more than likely you have some kind of runtime dependency on a JavaScript library so you may be using a scholar Library that's actually a rapper for JavaScript and you need to get those runtime dependencies into your application somehow so I was pretty intrigued by this shiny thing called Web jars um because the promise is you can download your JavaScript libraries from a maven or IV repo just like any scholar dependency and then the SBT plug-in will take that and produce one Javascript dependencies file and all you got to do is include that in your HTML and you're done so another confession though um I probably spent half of the hackathon trying to actually get it to work and to its credit it's gotten a lot better in the last 6 months but I still find that there's some practical limitations that are sort of preventing me from using it um you do have to think about the versions if you're if you're using um jQuery for example very popular JavaScript library that you might have conflicting transitive dependencies um non-javascript assets that you need to include let's say you use bootstrap and you have your bootstrap CSS you still need to make sure and include those um all of the dependencies do end up in one file so if you're not building a single page application that might matter and there's a little bit of awkwardness in the syntax because you have to tell SBT one time hey you know please download this and then you have to tell scholar JS umt plug-in please extract this Javascript file from the jar and include it in the app um so I actually after spending a lot of time with with trying to get that working I ended up going back to the oldfashioned way which is how would would you do it with any other JavaScript application um and I have not mastered the the Bowers and the grunts and the required js's of the world so I just you know take my library and I include in the HTML and that works fine and I haven't thought about it since so at this point I have gotten my Hello World building and now I want to take this application that I originally built in JavaScript and start showing proof of concept in schola JS so let me just quickly show you that application just to give you an idea of what what we're trying to do here um as I mentioned we're building an AI algorithm and it's running a series of simulations and throughout it's spitting out metrics into Json so this is a dashboard application that will allow us to visualize these metrics so I select a metric and then I create this chart which is way too big for the screen um but what you're seeing here um is essentially a visualization that uses D3 if you're not familiar with it it's a JavaScript library that does very fancy mathematics that will spit out svgs and it has data binding capability to keep your um HTML nodes up to date as you're going um and it's a very flexible Library so this is this um example is just showing a basic line chart but we really want to do some more interesting things we want to show for example what is our neural net actually outputting so I can create okay this is not showing up very well on the screen but anyway I'm it's essentially showing a raster image of the output of the neural net so that's using the same building block so using D3 getting that to work is essential to our application so I don't really want to take the entire application at once because that's a little bit too daunting so I'm going to take one specific D3 example and and see if I can come up with a pro of concept so I'm going to try this thing called Force directed graph this is directly from the D3 website and you'll notice it's kind of this fancy graph that that moves around and does kind of cool things with these forces applied to it um now luckily the code for this visualization is already available on the website so what's the first thing I do copy and paste I take this code and I copy it into a Scola document and I basically massage the syntax until it compiles so what I end up with looks something like this um don't worry too much about reading all of this code I just want to make the point that the two look very similar the original Java Script I mean I'm changing vars to vowels there's a slightly different Syntax for arrays for example but it largely looks the same um so how does this actually work would be the next question let's go to the same code snippet and kind of examine what's going on here so I want to direct your attention to this guy at the top it's called js. dynamic. Global and this is a scala.js API that gives you access to root JavaScript scope and what that means is you can call any JavaScript API directly from here and it's the compiler will assume that those are available at runtime so I know what you're thinking at this point um runtime you know what what is the compiler actually doing here um well it's not doing much because I can basically call any method that I want so I'm going to add some bogus method call Homer do okay so I'm going to go back to my server and watch it compile um this is the fast optimization happening and let's see what happen when I load up this page right so here's the nice graph refresh the page what happens it doesn't show up what's going on um so I have to go to my console and figure out oh this function doesn't exist H so you know at this point you know this is kind of what what I end up getting working it works right um but I'm leaving this hackathon kind of going well what am I really getting out of this right and you know is this something that you know how can I Leverage The compiler really to do some of this work for me and not be not be running with Scissors all the time so yeah what about those types turns out schola JS has a facility for this and it's called a typed facade so in that Global example we saw you're literally calling the JavaScript API and fortunately facades allow you to literally call the JavaScript API in a type safe manner so essentially you define the idea is you define this trait it extends this special magical thing called js. object which signals to schol ajs hey you know the implementation will be available at runtime and then you fill in these implementations as js. native so the beauty of doing this is I have now isolated the dangerous parts of my code to this one little trait and then everywhere else I can use this trait um and then the other beauty is is that if you have a large JavaScript library that you're trying to call you can do it lazily because you only need to essentially Define the the methods that you're actually going to use so if we don't need Maggie we can leave her off so that's great um let me just quickly show you the code for that oh wrong one so um you know you won't get an idea of the whole code but essentially the point is this code looks basically identical so the only difference is if I try to do the same mistake um before I even leave my IDE I'm getting a red squiggly thing and it's telling me no that's not going to work so I've just saved myself a lot of time in hassle of debugging so that things are looking good um so far though I've just shown you a static visualization right we all know that any non-trivial application is going to have some kind of Dynamic Behavior or you know there's going to be asynchronous calls there's going to be data coming back from the server so let's actually see you know the next thing I want to do is see you know how can that actually work um so I come up with a really simple web page here and all it does is it essentially opens up a websocket and there's a counter that data that's being pushed from this websocket every second right pretty simple and and and what I want to illustrate here is how do you keep your application state in sync with your Dom and in the JavaScript application I was actually using angularjs for that so my first instinct is well how do I make angularjs work in uh schola JS land and it turns out that there is already a facade Library that'll let me do that so let's just kind of pull apart this um this example we'll start with the HTML I have to define a page and um if you're not familiar with angular the main feature that I'm using from angular is essentially the ability to bind your application state to your node which means embedded in your HTML you're basically referencing some variable from your JavaScript right so I have this thing called NG app which is my angular module that that's where all the all the information is coming from on this page I have this thing called NG controller which has a little bit of business logic and it has a little bit of State and then I have this placeholder for some variable that is going to be updated at runtime this is where the magic of angular happens with the data binding so how about the Scola side of that so I have there's a there's a few pieces here that need to all come together um first of all I'm creating this thing called um JS something that extends JS app which is essentially the bootstrap of your of your schola JS application and so the main method essentially runs when the page loads and it's going to do all of the things that angular requires you to do in the life cycle to get everything bootstrapped and registered essentially so given that I'm using a Scala facade for angular I can define a trait and eventually this trait actually extends that js. object thing we saw earlier so this is is really just me saying this this is the type of thing that I want to update from my schola Js application that's then embedded in the HTML um and then I need to define a controller so the controller has to have some name that matches the name referenced in the HTML and all it really does is it opens up a websocket and then every time it gets a new message it's going to update my scope variable and we're done so all in a type way sort of but we'll get to that in a minute um all the code I should mention that I'm showing you today are is linked from the slide deck so if you want to go follow along feel free when when the slide deck comes out okay so I'm actually feeling pretty good at this point because I have a basic prototype application that does visualizations and now I can fetch data from the server and update the the view um but I'm still kind of wondering can I do better right um how many of you guys were Java developers before you did Scola a lot okay so you probably had the experience where when you first approached Scala you were kind of tunneling on how do I write Java code in schala right and there may have come a point when you when you thought well you know there's all of these fancy things that people keep talking about in Scala and like how do I actually use that right what are the what are the actual patterns and I reached a similar point with scholar JS I said okay I got the JavaScript way working in schola but now how do I actually get the schola think think in terms of developing a Scola application that runs on the JavaScript runtime and so first I remind myself well what is the JavaScript runtime I mean so the key attributes or distinctions from the jvm JavaScript is single-threaded anything that takes time is going to be done asynchronously call back sort of a primary method of communication and also guess what guys mutable State yeah um so if if you if you're new to schol JS and you're you're about to embark on this journey I would recommend just take a deep breath and just just kind of give mutability a big hug because it's okay um and so so kind of thinking how do we approach mutability in schola um I looked at at what libraries out there and I got a very helpful pointer to one in particular um that works well with Scola JS and this library is called scholar. RX it has um three key components the first one is called VAR and it's essentially like a scolar but it's got magical powers whenever your application updates a VAR Scola RX will kindly then update anything that depends on that VAR so you kind of get this magic change propagation going throughout your application and then the RX is essentially like a subscriber to a VAR so you can treat it like a readon variable that you that can then react to whatever changes are happening to the the things that it depends on and then the third component is an observer and the Observer can then look for changes to any VAR RX and it has a code block that Scola RX will call for you whenever those values change just to kind of highlight high level how this works um at so at the top you have the squares which are the vs and then the circles are the rx's and so when your application mutates the VAR Scola RX will propagate that change through all of the green circles so everything gets updated like that um so that's kind of sounding pretty good for mutable State and application State concerns um but if I want to replace angular because I'm I'm at this point I'm thinking can I just get rid of angular um I have to do something about the Dom and if you if you notice that angular example had these stringly typed things that needed to match between the HTML and the Scala and I was getting sick of having to go back and forth between the two and debug you know well did I have a typo and you know what did I do wrong here so it was quite it was much better if it would be great if I could simply just do it all and type safe uh code and not have to go back and forth fewer moving parts so it turns out there's another scholar Library called skola Tags that has this capability you can create typesafe HTML and if you're running it on uh Scola JS it can actually render to Dom nodes that can then be further manipulated and it's composable and it also allows you to template and you can share it between client and server so you can spit out HTML either client or server so now the question is how do you put these two things together and here's the idea so your application has some VAR and JavaScript wants to throw a whole bunch of events at your application so wouldn't it be nice to just respond to that event by updating that VAR and then have the UI respond to that re you know react to that without having to dig into the Dom query you know get element by ID or what or class name or what have you and then go manipulate it right um so what we do is the schola tags Library allows us on the bottom to Define some tag this is actually an H1 tag if you didn't guess and it's it's in type safe schol code and so because of that I can now directly embed my VAR into that tag and now magically it's going to be updated so a little bit more detail on how that works so essentially this this little magic method called as frag is a is an implicit conversion that Scola tags that you know knows about and it will install this uh scholar. RX Observer on the VAR and the Observer now can take the node that's being rendered by scholet tags and manipulate that in whatever way necessary as a result of changes to this VAR so um now that we have this building block let's go back and look at that D3 graph that I showed you earlier and how you know how how did the code change going from using you know just a JavaScript facade to using Scola tags and scholar

RX um so I kind of realized at some point that D3 is this huge black box and it's doing a lot of stuff it's handling not only the layout but it's also handling the application State and it's doing updates to the Dom and so here's here's kind of the what I ended up with realizing that well okay I'm in schola JS I don't have to use D3 for everything I can use D3 for what it's good at so here's the side-by-side comparison of the code before and after please again don't ruin your eyes and try to read this um the main point I want to make is that the code on the right is smaller and it there's no green in it which means I don't have any stringly typed things in my code because when you're using D3 you have to put the name the actual name of your HTML nodes in these strings and lots of things can still go wrong even when you're using a typed facade um so let me just show you a little bit more detail of the code no this one okay sorry it's hard to read here um so the point I want to make about this is that what what ended up working here was rather than using D3 to spit out some SVG for me and be a big black box I ended up using Scola tags to spit out the SVG and now D3 is in charge of the layout and how does that how does it do that right because when it's Genera this graph that I showed you earlier these little circles are moving around on the screen now that's D3 Computing uh XY coordinates for these little nodes and guess how it's communicating those changes yeah through callbacks right um so I don't want to have to mess with callbacks here so what I ended up doing is taking um this callback and turning it into the RX from Scola RX so now instead of having this series of callbacks I have this one RX and its value is changing over time and so I can use the trick of embedding that directly in the Scola tag and now everything gets updated so everything's in one place and uh it's pretty awesome so let's go back as well to the websocket counter example and see what that looks like side by side Tada um so before we had you know HTML and a code we had stringly typed things and there were about four or five moving Parts um and now on the right to do the very same thing I have four lines of code so um that was kind of the big yay moment for me let's go look at that in more detail um so I applied the same pattern really here um I created this little class called callback RX and essentially it takes some series of callbacks that are happening and converts it into um you know an RX and so pretty much all I have to do at this point is embed that in my HTML and I'm done so so I want to leave you guys um with a few resources that really helped me along the journey so here's my scala.js treasure map um the first thing I actually did to learn schola JS was read a book called Hands-On scholar JS it's it's actually all online and available and it's by um Howie Lee and this book gives you the breadth of the schola JS ecosystem and it allows you to do a few Hands-On examples along the way and when you're kind of hungry for more and you want a deeper more like closer to real world application uh example I'd recommend the single page application tutorial by autocon so that one really gets into you know what are kind of the core patterns to follow um I also had a lot of little technical questions along the way and I was um I ended up using stack Overflow tag which is heavily monitored by the schol GS community so it'll benefit everybody if you ask your question on uh stack Overflow I also joined the gter channel and I have to say it's a very strong Community with really uh friendly folks who are willing to answer questions um so I definitely need to thank all of these guys for helping me out along the way I had a lot of kind of dumb questions along the way um and then the other thing that I learned when I was ready to shift from the JavaScript mindset to The Scholar mindset that whenever I needed a library it turns out that this one guy OE leap was probably has probably already written it so I definitely encourage you to just look at look at his um GitHub page to get an idea of of you know kind of the beginnings of a Scola JS ecosystem um also I have put my application on GitHub so the the whole source code is available um I've kind of gotten some interest from folks who are interested in using D3 in scholar JS and we may end up starting to build a library around that so right now the code for the D3 stuff is all kind of stuck um in this repository but if you're interested you know please come talk to me and we can talk about maybe pull pulling out a library for that so I want to thank everyone for your attention today and uh want to open it up for questions we do have a few minutes [Applause] left all right questions oh intell yeah question oh describe how I embedd the RX into D3 the trick is I didn't um I I use Scola tag so essentially the trick was not using D3 for any Dom related activities I was using D3 to compute to do the the actual layout so it would compute where should everything be on the screen um but then I was I was sort of pulling that out of uh D3 using the rx's so that I can embed that into the Scola tag hopefully that made sense3 noidea it does not it does not it's so all I'm doing is I'm using that for the layout piece of it so uh in the backs Cod you that oh okay so the question is did I ever have to readed decompile JS code and do that for the debugging um that's a good question because as a matter of fact um the Scola JS compiler will generate Source maps for you um so you can literally debug your Scola code from the browser and can set a Brank point in the scholar code um so it's really good support yeah that's yep yep it just works I've never had to mess with it anything else okay thanks everybody [Applause]