SF Scala, James Ward: SBT-Web
Recording: SF Scala, James Ward: SBT-Web
oh well okay so i'm james ward i'm a developer advocate at typesafe and tonight's talk is going to be about spt web it's a brand new thing that we've been working on it's been mostly been worked on for play but it's really uh generally applicable so if you're not using play if you're doing anything related to client-side assets client-side programming in your scala apps then this will be relevant to you so um so that's what this is about tonight feel free to stop me at any point and ask questions um oh i have a question alexi what time do you want me to wrap up actually i think i'm pretty flexible okay so 10 o'clock is cool all right i guess if the beer starts coming in then we can okay so we'll go for beers or something at some point record okay it's recording um yeah so feel free to stop me at any point and ask questions i'll be writing uh trying to write a bunch of code tonight and and uh we'll try to make this as interactive as possible so let's start out with a couple slides and then we'll get into some code so sbt web uh it's this new tool new new set of tools and really what it's about is creating some sbt plugins that allow us to have grunt-like functionality inside of our spt builds so we support the node api and also web jars and we'll talk more about what those are in a little bit ultimately it's an asset pipeline and then has a few other features that we'll we'll see as well later so what does that actually mean there's a bunch of different parts of asset pipelines uh there's parts of asset pipelines that do source linting and this is all for client-side assets so this whole thing it's all about the stuff that runs in the browser it's all about our javascript our coffeescript our less our sas our all that whole side of our development what what's been happening what we've seen is that a lot of people are using play or finagle or other scala libraries to do their services and then they use the javascript tools to do the client-side stuff and that's that's fine if you want to go that route you know that's that's going to work just fine but what we want to do is have an option for people who want to have their whole tool chain on top of the jbm and so the way that we've done this is we've created this this set of asset pipelines so first there's source linting so this is being able to lint our javascript or coffeescript whatever this is being able to take source files and create executables so in executable in the web world is obviously javascript so we can take source files whether that's javascript to javascript or whether that's uh whether that's coffeescript or typescript or scala js whatever those things are we're going to transform those into the executable that the web can actually run then there's uh then there's on the other side those are the the source ones there's asset pipeline tools that will take executables and then do some transformation to those executables so one of those is being able to optimize the executables uglify is one of the tools that does this so this takes our javascript and then turns it into a minified version of the javascript that's an optimization of post compile optimization there's fingerprinting so we want to be able to have an efficient way to fingerprint our assets and this can be the way that this is usually or the reason why this is usually used is that we want to fingerprint these assets so that we can have an etag header that is some value that's unique to that version of the asset and we use that etag in both the http header but we also use it in the url so that when we want to change the version of that thing we now have a new url to the new version of that thing and that's important because if you have the same url then you can't put far future expires on these things and so uh really changing the url is the primary way to be able to tell the browser put this in your cache and never ever ever again ask for it it's called far future expires you basically tell the browser like like this content it is never going to change and so the way that you change it is to change the url to it so that's what we use asset fingerprinting for concatenation is another example of this where i have you know 100 javascript files and i don't want the browser to have to go make an http request for every single one of those instead i want to be able to combine all those together into one javascript file or maybe a couple javascript files so the browser only has to make one request that's going to take longer but it's going to ultimately take less time because you know there's a limit to how many concurrent requests the browser can make so that's concatenation so that's the job of the asset pipeline is to do that and then not really part of the asset pipeline but something that's part of this tool chain is is testing as well and with testing we're able to actually in sbt run tests in our javascript environment so actually execute javascript tests just like we're executing our java or scala tests as well so those are the primary things that the asset pipeline is is doing for us and what we're going to cover tonight before we get into the details of how we actually do that with sbt web any questions about what an asset pipeline is and what it's doing okay so svt web what it is it's an asset pipeline on top of sbt play has had an asset pipeline in it from from the beginning i think and the but the problem with it is that it wasn't very flexible uh it was hardcoded into play uh and it actually used the the compilers that it used the tools that it used were actually like forked versions of the less compiler that everyone uses in forked versions of the coffeescript compiler and forked versions of the coffee the the closure compiler and so play was maintaining all this stuff and some of you have probably had the experience in play where you try to use play's asset pipeline to compile bootstrap is the one that we've heard a lot of people complain about and the problem with that is that the the less compiler has evolved ahead of what play has done with its less compiler and so most of the time bootstrap is being compiled with a newer version of the less compiler than what play has in it and so bootstrap from less source won't compile in play and so to address that that's one of the things that we created sbt web for was to allow you to have a lot more control over this asset pipeline not hardcode things into play but make this a lot more plugable and then also make it something that's usable outside of play so sbt web even though this is going to be it's going to replace the old asset pipeline in play and play 2-3 this is usable totally outside of play there is no dependencies on play from sbt web so it also benefits some from some of just the foundational features of spt spt has really great incremental support has really great parallel support and so spt web just inherits those those traits automatically so that means grunt is actually single threaded spt web is actually multi-threaded by default because spt is multi-threaded and can run things in parallel so we can actually compile your coffeescript and your less at the same time now in sbt web so that's a nice feature of being built on top of sbt something that's actually really important to this is that you can choose the back end that you actually want to run these compilers on these asset pipeline pieces on so you can either choose to use the node native stuff so if you're using less you can choose to use the native less c that's on your system if you choose to install that through npm you can choose to go that route the default route is actually to use node on the jvm so there's actually an implementation of node on the jvm called trim and so we actually run node on a implementation of the node api on the jvm so that means we run the actual less compiler and the actual coffeescript compiler on the jvm through rhino now but we'll run it on naz horn in java 8 as well but this allows you so that you don't have to install these additional dependencies on your system in order to be able to use this asset pipeline and then you can choose if you want to use the native one or the jvm one today on rhino the the the trireme one is probably going to be a little bit slower in terms of a single compile than than the native one because it's running through rhino and rhino is is interpreted and pretty slow so that's one of the downsides but if you want to have better compile performance on a single file then switch it over to the node native ones and you're good where this actually becomes a really big deal for for me is we have for typesafe.com it's of course a play application we have asset compilers and all sorts of stuff we uh we today or we were using a node native compiler to compile our css we were using stylus and the problem with that is that in order every time we stood up a new ci server we'd have to then install the stylus executables and node tool chain on those servers obviously it's not a huge pain but but definitely was something that wasn't real fun but then we also had to do that on heroku and figure out a way to do that on heroku as well because heroku you kind of have to tell it which environment you want do you want the node environment or do you want the java environment so just having to like having a full tool chain on the jvm definitely makes ci and deployment a whole lot simpler so so that's one of the reasons why we wanted to be able to provide the option of either running on the jvm or with the native libraries so then one of the other benefits of sbt web is that we use a project called web jars to define our dependencies on all of our asset compilers web dryers let me show it to you real quick so web jars and sorry that i'm on windows i rebooted so i can get some screen capture uh here because i'm usually on linux and linux doesn't have great screen capture so so i'm on windows but i think everything should work okay we'll we'll find out so web jars is what it does is it packages up public javascript and css libraries into jar files and puts them in maven central so now that we can depend on these client-side libraries just like we depend on our server-side libraries so if we want to use less we'll see that there's actually two different less ones here one is there's actually two different distributions of less one is for the browser and one is for node so one's written against the browser api one's written against the node api so we actually have two different web jars here for for less but this this file let's actually go look inside of it this has in it the less.js and so now not with this one with the other one but we can actually now run that javascript on the jvm through trireme through rhino and run it on the jvm or you can use the native lessee if you want as well but we can actually take this exact javascript file i can specify a dependency on less version 1.7.0 and now i use that javascript file that came straight from the less providers and uses that as the asset compiler and what that means is that this is just a standard dependency in my project so if play ships with with less 170 and i want to upgrade to 171 all i have to do is override that dependency in my project and now i'm able to upgrade the version of less that my asset compiler is using outside of a release of play and so that provides a whole lot better mechanism for managing this kind of stuff than what's currently in play so web drivers you can also use these for the client side as well and and we'll see that a little bit later but uh in the context of sbt web we use it for the actual compilers all of the compilers for all the asset compilers today are being written in javascript because they're being written by the web folks and so that's why we need this layer to be able to actually and they're being written against the node api so that's why we need this layer to be able to actually execute javascript on the jvm if we don't want people to have to install additional tools so yeah so that's that's really the core of what spt web is about uh any questions about that stuff before we go on yeah uh compatibility so in terms of performance you said that native uh is faster and single thread what about compatibility across these implementations uh compatibility should be the exact same because we're we're using trireme which is a node implementation on the jvm and they uh they actually have a compatibility chart for the apis that they've implemented and tested in node and i think they're getting pretty close to like 100 compatibility with implementing the node apis so we haven't seen any issues there yet um so so we're compatible there and then we're fully compatible because we're using the same source for all the compilers that the node versions use so that less.js or it's actually less c.js that that javascript we're running that same exact javascript for spt web as node runs for running the less compiler when it runs through node so it's the same exact source code and that's really important because we don't want to like get into the game of like trying to write a less compiler in scala like all this stuff is being done in javascript let's just embrace it use it as libraries and so as long as that node uh that node implementation works then then this all works and if it doesn't work switch over to the native one and you're okay and it's really just an sbt uh toggle to to turn it from the from our um implementation on trireme over to node so it's just a build setting yeah good question other questions okay so that's that's the the basics of spt web so today uh where we're at with spt web is we've just released uh milestone two a is the latest versions of all these different spt web plugins and so these are all just standard sbt plugins you can add them into any sbt project and we've released so 1.0.0 m2a is the current version play 2.3 m1 milestone 1 was just released a couple days ago and that brought these into a release of play so if you want to try this out i'll show you an easy way to get started but play two three will will have all these included and you can try them out now on any spt project or with play two three so uh the the ones that we have there today our first js hint so we use js hint as a as a linting tool for javascript there's coffeescript support there's less support digest the digest is what gives us that the etag creates actually an md5 file for every asset and i'll show you that in a little bit gzip does a gzip on all of our assets moca this isn't really an asset compiler what it does is it it's the plug-in for sbt that runs our javascript tests as as part of our sbt test so that's spt moca moca is a javascript testing framework and then rjs is abbreviation for require.js so require.js is what uh now gives us concatenation uh and uglification the the minification and then the ability to specify client-side dependencies and have those those managed on the browser for us through required.js so those are the current ones that we've been working on the one of the really great things about the whole sbt web infrastructure is that it's now super super easy to create your own asset compiler it's really a few lines of code and you can create your own asset compiler you know we need to create one for typesafe.com for stylus and it's going to be super easy once we do that it's really because we're just using the stylus asset compiler that's already out there and putting a little spt wrapper around it so the community definitely welcome to to work on other asset compilers uh so there's some other ones that that we're working on the community is working on one is a standalone uglify one so we want to be able to outside of require.js do the minification scala js is one of the exciting ones we want to as part of this asset pipeline build to compile scala code through scholar gs into javascript sas and stylus or some others and there's you know tons of other options out there for what people could do for uh for asset compilers so that's where we're at today and before i show that to you any questions about that or comments or other ones you need yeah i tried playing this this past weekend trying to use it too yeah i didn't have any it should work with play two two um what you what i think you actually need to do is turn off all of the built-in asset compilers in play 2-2 so there's some parameters that aren't well documented that you can turn off the existing ones and then it should work if if you if that doesn't help post a message on play framework and christopher hunt the primary developer behind this can can help you out with that yeah yeah but it should definitely work with play two two it should work though the one trick to this whole thing is it does require a new version of svt uh that may may have been the problem but so it's uh spt-013 5m2 or something ridiculous like that so it's a new version of sbt yep yep yeah and i'll show you some web dryers and stuff later cool great other questions about any of this so far okay let's see uh mention that all right so let's actually create a new app and add spt web to it so to do that i'm going to use activator we just released activator 1.1.1 so that's the newest release of activator activator you may have seen in a recent announcement about play activator is going to become the primary distribution for play in the future so the what activator is is it's an sbt launcher just like the play commander is an spt launcher and spt has an spt launcher so we have three svt launchers we're trying to consolidate that down into now two spt launchers so activator and the spt launcher which activator is an sbt launcher so there's activator as an spt launcher which allows you to an activator you can launch sbt and do spt tasks just like normal svt and then you can also launch the ui for activator so there's a ui mode and a cli mode if you don't want to use the ui that's cool you can just use the cli so we have people that prefer both up to you which one you want to use so i'm going to launch it first into ui mode and so i'll just run activator and then i'll create a new project in activator so i created a new a new template activator has a bunch of templates that you can get started with and let's see how good our search is there it is cool so i created a hello play framework 2.3 uh scala one and so i'm going to use that template to start with here so this one just has all the pieces already set up all the the spt web pieces so it's going to take a minute to uh to get that set up oh and i i was doing this on linux before so now i'm gonna have to download some dependencies so it's gonna take a little while uh so while that's going let's go back i had a nice primed iv cache when i was on linux but not anymore so to set up to set up svt web the first thing we need to do is add a plugin to our sbt build so each of those sbt web plugins that i mentioned each of those you have to add independently and i don't know if we're creating like a wrapper one around those or not if you're always going to have to add them independently not sure what we'll do and and play with that but outside of play you'll certainly have to add each of them that you want so if you want to add the sbt coffeescript one then replace blah with coffeescript and put it in your project and then plugins.sbt file so there's a new feature in sbt-013-5 which is currently in milestone two that we're taking advantage of in sbt web and it's called auto plug-ins and auto plug-ins are a cool way where we don't have to bring in all these settings and have a bunch of stuff in our code to use an sbt plugin anymore and it's a really nice new feature in sbt so we're taking advantage of that in sbt web to make it easier to to set these up so really all that you have to do to enable one of these plugins is add it to your project and then plugins.spt and that's it and then if you want to override some parameters you can but you don't have to let's see where our build is at still downloading dependencies good thing oh now we're loading project good there's fast internet connection here so once that's done so we in activator you can download activator in two different ways you can download are actually three different ways you can download activator as a full distribution which includes a bunch of dependencies in a primed ivy cache right now those dependencies are like play two two one and spt sbt 013 0 and scala 210 1 or something like that so a little bit older versions of of some of those dependencies but so if you download the full distribution of activator which is like 250 megs then you get a primed cache with all those i'm using newer versions of things so my primed cache is not being used for any of this so now i have to download the whole internet so there's also in this case what you should do if you don't want any of those old dependencies you don't want that primed cache or it's not useful to you then download the mini distribution of activator it's a meg and then that after that download downloads all of your dependencies after the after you start it up so two different ways and then another way to do it is you can go to a template and download it what's called a template bundle and a template bundle is the template and the one meg activator launcher so those are the different ways to do it again in that way it's going to have to download all the dependencies from maven central yeah there's very question of this so i'm used to just using sbt launcher yeah and there is a nice wrapper called svt extras on github which lets you specify this pt version so what kind of confuses me with activator is that you know you're kind of like a dumb user downloading something into your home directory yeah suddenly there are multiple versions of the activator so let's say i already have an activator an older version what should i do if i want to use a new one should i just discard the old one is there a way to manage multiple installations what's the right way to do it yeah so there's there's a couple different things there one is the actual sbt version if you have a sufficiently new enough version of an spt launcher it actually will download the version of sbt that's specified by the project so that's been the case since like the i don't know 0 12 something launcher is it actually looks at your at your project build.properties and then downloads that version of sbt to use so so uh so you shouldn't have to have multiple versions you shouldn't have to have the actual sbt launcher for the version of sbt that you're actually using those two can be decoupled and so that's how activator works as well is that right now i'm using um you know activator launcher 1.1.1 and when i launched it you saw that it started pulling down spt 0 13 5 m2 or whatever so so it was able to know like hey i don't have the right version of spt i'm going to go download that version of spt and then bootstrap into that version of sbt yep so then for activator itself so that's the other aspect is the the s the activator launcher uh it's really just the spt launcher so so that's you know you can run spt commands based and that's wrapping the spt version that your project actually is but then for the ui of activator we actually now us i think since 1.0.7 we started auto upgrading the ui part of activator so if you have activator 107 and you launch the ui it will actually go check to see is there a new version of the ui available and if so then it will pull the new ui down which is just some dependencies and pull those into your iv cache and then launch the ui for the latest version of activator so that's been the case since i think 107. so so we want to make it so you're not always having to re-download and upgrade these things so really just that launcher piece you should have to upgrade very rarely um unless we screw things up which we have so um so there are there have been bugs that we fixed in the launcher itself the most recent bugs around this have been on windows and have been related to proxies and those sorts of things so for most users they shouldn't have to upgrade the launcher very often does that answer that okay cool let's see how our how our build is going still loading project come on let's try this again 2.3 actually maybe it created it let's get check oh here's a cool little feature of of activator i'm going to stop the the activator that i launched here uh you can once you create a new app from a template with activator it copies the activator launcher into that project and i i know it's probably bad in some ways but i just put these files into the the github repo to my git repos so that any user can just clone the git repo and then launch activator straight from there they don't have to download or install anything so and then if you uh if you launch activator from within a project directory then it takes us to the activator ui for that project instead of taking us to the the home screen so let's see how we're doing hopefully my demos don't fail compiling project definition sbt lock all right we'll give this thing a minute and see if it works if not we'll just reboot into linux any other questions while we're waiting for the sbt lock we use iv underneath the covers today with with activator and spt and it's been a little bit of a source of of pain in this world actually you know what i'll bet it's not actually waiting on that lock i bet it's an erroneous lock uh i would love for ivy to go away but it's going to be a lot of work to replace it let's try to delete our locks locking i thought we were reactive what's what is all this locking stuff right all right let's try this one more time this doesn't work we're going into linux doesn't always work on a projector but it the iv cache usually works trade-offs right so we'll give that a minute um i guess while we're waiting for that let's see uh that's more sbt stuff so um any other questions while we're waiting to see if this works okay any questions about play play two three other features in play t3 while we're here yeah is it gonna be uh easy to upgrade from a project yeah it should be so things like the asset compiler should just work if you enable the new asset compiler so that part should be pretty straightforward we have some new apis and some apis that were deprecated in 2-2 and will be removed in 2-3 so the primary api change is around a simple result so we created this intermediary uh thing called simple result which we deprecated in 2-3 and simple result or sorry we deprecated into two and two three it'll actually go away and be replaced by just result um or maybe it's uh maybe we're not doing the deletion yet but so that's one api that will change is we're doing this transition from an old api to the new result api so that's something that that could change there's a new actor for doing websocket communication that's pretty cool so that's coming up um there's uh what else is is coming in two three the asset compiler stuff has been a big one there's been some changes in some of the java apis around java 8 and better supporting lambdas so if you're a java developer then then those will be useful if not then that's okay so yeah so those are the the primary changes that i can think of is java 8 and the client side asset compiler stuff were really the two big focuses of two three yeah whoo hey we're in a project finally cool okay so here's the activator ui for my my app that i just created and i'm going to actually open this up in intellij activator does have a little web-based code editor i don't ever use it i use intellij so i'm going to open this up here in intellij activator does actually run the app so we've got the code browser we've got compile output which is failing oh no demo fail we'll see if restart let's see if this works if not i'm going to reboot so um so activator it'll run the project run the compile run the test and then we can also inspect the app the inspect what it does it doesn't actually work yet in two three it doesn't work on a milestone release so it only works today on two one and two 2.2.1 2.2.2 so inspect instruments our applications so that all the requests that are coming into our application we can go in and look at and see what the request headers were and that sort of thing and then we can also see information about our actors what actors we have in our system how many messages they're receiving that sort of thing so but it won't work on this app because this app is a 2 3 app and we're still resolving assets we're going to give this one more one more chance to do the right thing and then we're going to give up go to linux okay so every app every to every template in activator comes with a tutorial so when you're in the ui you can read through the tutorial learn about the code this particular tutorial i haven't actually written yet so there's not a whole lot of information in this one but there's some other ones that have a lot of really great information in the tutorial so you can check out the tutorials learn about the code and that's uh that's good um looks like we may be getting close to resolving artifacts we're resolving oh i can go start looking at the code though now oh spt locks damn you oh man all right we're gonna free up that lock so that we can actually go look at some code here let's go see resolving resolving artifacts hey there we go we're moving forward downloading the internet okay this is why we want primed ivy caches before we do demos yeah good for you how do you launch xbt on your ci environment and on heroku do you install it as a dependency ahead of time so on heroku they have a build pack for spt and so the build pack is uh the build pack has the spt launcher knows how to get the spt launcher and so so they do that for you on my ci system you uh there's two ways to do it one is that you could install i think jenkin jenkins does have an sbt plug-in and so that brings in the spt launcher to jenkins and then you can just set it up to call the spt launcher or i haven't tried this but you should be able to use the activator spt launcher inside of jenkins as well i haven't tried to do it that way i've always used the sbt plug-in in jenkins but it should work either way yeah yeah oh yay our project is finally open all right cool okay so we're we gotta open project next we'll see if we can actually run it and i'm going to change my font size real quick so we can actually see oh it's on the bigger font that's good okay now let's see if we can run this thing let's try one more time we can start looking at code okay so let's go take a look first at my plugins.spt file here so plugins.spt this is where i define first the play plugin so the play plugin is that spt plugin poorly named i guess but that's the spt plugin for play and then we define all of the spt web plugins that i want to use so you'll see the required js coffeescript less js hint digest gzip and mocha so those are the different plugins that i'm using so now if we go to my main project build definition we'll see that i have a few dependencies in here one is on the web jars play library we'll look at that in a minute i've got a dependency on the bootstrap library i'm using bootstrap for for the ui in this app and then i'm also specifying a dependency on the required js dependency and i may not have needed to do that it's i think the sbt plugin may have actually brought that one in so so i may not have needed to do that then you'll see down here at the bottom of the file this is where we bring in all the settings for a play play project into this app so if you're doing java then it would be play job if you're doing scala then play scala this replaces the uh the play java settings and play scala settings that you usually have in your build.sbt file so so this also brings in the sbt web settings so if you're using sbt web outside of play then you actually won't add add the plugin for play you'll add the plugin for sbt web the play plugin here the settings that brings in also the sbt web settings so so it wraps that so that's our that's our build settings that's that's all that we need um any questions about the the build the sbt build settings okay i just have one question about the idea i know that idea has some play um plug in it for itself i wonder if that's useful if that's actually interacting in any way with this yeah so the the play plug-in inside of intellij it um it doesn't do anything as far as i know with sbt or any of the the resources or any of that stuff where that where that plug-in helps is on the routes file so it understands the route file syntax it understands the scala template syntax so it understands the things that are specific to play but it doesn't do the sbt stuff what does the sbt stuff is the scala plug-in so when you install the scala plug-in intellij it knows what an sbt project is you'll see that i opened up this spt project without having to generate any project files that's a fairly new feature in intellij and the scala plugin so i can just go to any spt project open it up in intellij it pulls down my dependencies for intellij or resolves them for me intellij and and sets up all the the project definitions for me so that's that's part of the scala plug-in so in this case there's not anything special yet about the play plug-in for what i'm doing yeah okay so we've got any other questions about build definition stuff okay so that's our build definition uh we'll show you let's see yep it is 0 13 5 m2 is the sbt version that we're using and let's go see if we're able to run this time oh could just not work on windows we will try this a different way let's try the command line all right let's hope this works this time okay so that's our build definition let's go see now the um routes file nothing real interesting here if you're not familiar with web jars we have a route to uh there so that we can get to our web jar assets so this provides a way to get to like bootstrap.css and those and jquery.js and those sorts of things so that's that's our routes nothing real interesting in here um our controller so far nothing nothing real interesting in our controller and we're gonna stay out of the controller stuff tonight so let's go check out our views so here's our our main index page that we're using and then that's calling main so here's our our main page so let me show you how the the web jars work so in this case what we're doing is we we've have a dependency on bootstrap and so i have a line here in my template where i'm saying all right give me a style sheet and i'm going to use this web jar assets controller so that's a reverse route to the webdriver assets controller and then i'm calling this webdrawerassets.locate the reason for that is that the actual path to bootstrap.min.css is actually actually has the web jar id which in this case is bootstrap and it has the version of that library that i'm using so it was like 2 or 3.1.1 or something like that and then in a css directory inside of that there's the bootstrap.min.css so i'm using webjar assets.locate to resolve the path to that that thing the reason why i want to preserve those urls with the version numbers in them is that those version numbers essentially become the way that i can use far future expires on this content so web jars are set up to be able to be cached forever so a web jar asset you once you once the user loads that down onto their machine then you can put a far future expires header on it and the user should never have to pull that down again so that's why we leave the version numbers into the the web jars and so that we can get that behavior but we don't want to hard code those version numbers here on our in our paths so we use that locator to be able to get us the path um and we web drivers recently added support for required js so i think maybe later we'll we'll get into some require js and web dryer stuff but you can use webdrivers with require.js and then we also recently added support for cdn for having all these on a public cdn so you don't have to set up your own cdn for your web jar assets and it's really easy to toggle from pulling these assets from my local play app to pulling them from the cdn so that's how we set that up you'll see i'm loading in bootstrap.min.css and jquery and so that's that's my main app let's see if we were able to actually run this thing no we're still locked on the iv repository gotta love downloading the internet um why why it's like it's like it's uh yeah they're like competing against each other right oh man ivy cash as maven caches before ivy caches go away lock file i wish we could make iv reactive right all right all right so let's start actually using some of the the uh features here of sbt web so to start let's start with um let's start with the linter so i'm going to create a javascript file and we're going to have that go through the linter if we can get this app actually running so so let's create a new package here so by default in play all of our assets that go through the asset compiler go to app assets and you can change that it's just an sbt setting if you're not in play then it's source main assets is the the main source directory for your client-side assets so let's create a new file here and this will be fu.js and then you know this will just be our standard javascript source oh and it's not going to be very exciting if we can't actually see it run so i'm going to reboot into a working operating system with a good iv cache and that's not going to be fun but bear with me sorry the video is going to be sacrificed for a working demo that's okay so one second can't have a milestone product without a failed demo right all right so we'll reboot that so once we get this working what we'll see is that we can write javascript code and that javascript code if we have added the sbt js hemp plugin to our project then it will link that javascript and tell us if there are any javascript errors super exciting good thing linux boots in six seconds here we go let's go back to my project here all right i'm going to expect a whole lot of like cheering if this app actually works this time resolving dependencies please yay cheer we have a working out right yay thank you linux saves a day okay let's go load this thing up here and then we'll get going okay let's go to my presentation all right and now intellij indexing indexing index faster okay all right here we go now let's go create a new file again food.js and let's do some bad javascript what's some bad javascript look like is that bad javascript let's try it out yay okay there's the linter so the linter is telling me hey that's not valid javascript and we see it in our browser just like we see all of the other errors that we see in a play application so so we've got linting working that's great let's go let's go fix this and verify that everything recompiles so um i don't need a function let's just do alert oh and i probably need a semicolon gosh hate those semicolons right i hate writing semicolons in javascript now that i've been doing scala uh i didn't see my alert oh i didn't import it uh i didn't use it so let's go actually put this thing in a file here so let's go script and then source equals and then i'm going to use the reverse router so routes dot assets dot at and i'm going to tell it to load foo dot js right so save that and reload and if our browser lets us display alert messages there we go cool okay so now our javascript is obviously working it's in the page so that's the js linter not not super exciting or complex but definitely if you're working on a large code base of javascript essential to have that kind of quality assurance tools okay so that's that um questions about the linter before we go on okay let's go on to the next one so there's linter so the next one is a coffeescript one so as you probably guessed to add coffeescript i've already added the plugin to this project so now i can just come in here and add in a index.coffee file and let's do some some jquery here right and log something out using jquery on ready function call there and let's go back to our template and now let's use index.js and hit reload and if we go inspect there we see our asdf that we just outputted from coffeescript so now that coffeescript is now part of that asset compilation process just like the rest of our code so let's go back here and let's make a what's what's invalid coffeescript does that no at signs are valid um percent signs well it's invalid that's valid um there we go okay so now just like with the linter just like with our back end code we see the the errors there in the browser so that's the coffeescript compiler um one thing that is different about play23 and spt web in this respect is that i don't now right now have a minified version of this file uh because in order today in order to do the minification we need to use the required js plug-in and i could do that i could set up require.js we'll get there but uh and we don't yet have a standalone uglify spt web plugin but we will so so i don't get the minified version of this file until i run it through require.js right now so that doesn't exist yet so that's a coffeescript plug-in questions about that okay pretty easy okay so next up let's make sure we're still working here okay good so next up is spt less so super simple right that we can go in and create one thing you'll notice this is different from play uh before this is that i don't have to actually put these files into a javascript package or a stylesheets package it looks for the asset compiler by default looks for anything in the assets directory that has the extension that we're expecting so so it doesn't have to be in in the subdirectory format that we required in play uh before two three so let's create our index dot less and let's add you know body tag background color i'm very creative with my color schemes and changes let's go use that style sheet here so assets at and then what i call it index.css and i need some quotes does that look right oh wrong what did i do wrong index.css there it is background color oh body background color that's right rich why isn't my css working characters what is that oh thank you why didn't the why didn't the less compiler tell me that nice catch i owe whoever that was a beer that was that was a good one there we go now we see a red background cool so anyways that's the that's the less compiler it should be catching those those color mistakes right what kind of compiler is that um so that's that's the less compiler any questions about the less compiler okay pretty straightforward okay uh spt digest so this is the one that's creating our md5 we can also create i think sha1 files that contain the actual md5 or sha-1 signature of the file so the default is just md5 and then you can enable sha-1 or disable md5 if you want the way that we see this is to if right now if we look in where these files are going so if we look in this is in target web is where all of our assets are being managed this is like the the classes manage classes directory for my assets and so you'll see that that right now there's just basically the the standard stuff there uh you'll see there's also the web jar contents uh spt web extracts out the web jar content so that i can use those as well but you'll see that there's no md5s there's no gzipping there's none of that yet and we actually only run those tasks when you go into a production mode or a production distribution so the way that we can do that is to run uh activator and then let's uh we could run start or we could run stage or we could run dist are the three tasks that we'll we'll do that let's run um let's just run stage so we can see those those files so stage what stage does with uh play is it creates a stage distribution so that's the everything that my application needs to run somewhere else it's all my dependencies it's the start scripts and it's my application all compiled up and all my assets compiled up and so it it creates that and so if we look into target web actually let's go do that find again now we'll see that now that that um the digest plugin now ran on all those files and we can go see now that here is the md5 signature for index.css so we write this into this separate file and today we don't actually use that for anything but what we can use it for is we can use it to change the urls for our static assets is one way that we could use that and the static asset the static asset controller and play will use these to get out the md5 signatures for the files so that's one way that we'll use it you could use this as part of your build process as well people do different things like build app manifest files and those sorts of things based on these so so there's a lot of different things that we can do now that we have all these signatures there so that's yeah question question like one of the obvious uses for this would be to see that your site is not defaced that all the assets are still at the published version is there anything in play you can use basically enforce that the assets still have the signature um you know i don't think there's anything in play what we could build on top of this is the web has a manifest file and a manifest file i think can do what you're talking about i don't know a whole lot of details on it maybe somebody knows more but i think that with an app manifest file we can generate the manifest file and then do verification based on that possibly i don't think in the browser there's any way to maybe there is some way to actually pull down an asset and then do the md5 on the client side of that asset and then compare that against this file so there might be some way to do that interesting to explore does any know anything else about manifest files and what they can do i don't know a whole lot about them okay so that's the digest plug-in any questions on that okay um next up there's the gzip one and that one i've added to my project and you'll see that yep there are the gzip file versions of all these files as well so the one of the main purposes of this is that services like cloudfront will if you have the gz version of your file then it will use that as your st as your gzip version of that static asset so you could actually take all these files upload them up to s3 put cloud front in front of them and then have cloudfront serving both the non-gzip and gzip versions of those files based on what's been generated here yeah so if you if you do that using cloud or some other cdn is there any like kind of cdml integration either at the you know whichever part of the tier is that's left as an exercise to the reader we could easily write an spt plug-in that when we do like a play dist would take your assets and upload them up to s3 something like that let me show you how we actually do this for for typesafe.com um because it's it's the way that i prefer to set this up but it it's it's certainly not the only way so if we why do we have so many errors on typesafe.com rich rich is uh our web developer on typesafe.com here here my whole demo is gonna fail because we've got all these oh it's all the head requests interesting okay so what we do on typesafe.com is uh when i'm when i'm developing typesafe.com when rich and i are developing typesafe.com locally we're using web jars and all of our static assets are being served out of the play out of the local play application so my jquery all of my images everything is being served out of my my local play application and then when we deploy this onto our production server i i there's a setting there that says don't get these static assets from typesafe.com instead get them from our cdn and so if we look at jquery here the url to jquery is our cdn which is on cloudfront and then it's the same url uh the rest of it is the same url as the as what's available on typesafe.com so if i copy this url here here i can get to that resource right but in this case it's actually pulling that resource from from the cdn so it's pulling that from cloudfront what i've set up is sorry go ahead is the recommendation that you configure your cdn to fall back so what what i've set up the cdn in this case to do is the cdn the cloudfront it uses my server as an origin server so it uses typesafe.com as an origin server so when a request comes into this url to cloudfront slash webdrivers jquery whatever it if it doesn't have that asset on the cdn then it goes to typesafe.com to get that asset and typesafe.com has the far future expires headers it has the etag it has all the right information it has the gzip encoding it has everything that's needed for cloudfront and then after that first request then cloudfront then caches that asset on the cdn and so then there's not going to be any additional requests to get that asset from typesafe.com so that's the way that i prefer to do that because there's no additional deployment step there's no like copying files to s3 uh that's that's uh the simplicity of this model is why i chose that but there's obviously a lot of different ways to do this sort of thing other questions about that okay oh yeah go ahead uh yep you definitely could um today so uh i don't do that here there's some trade-offs to that one is you get a whole lot better cash in if you're using the one that everybody else is using but the downside to that is that then i don't feel like i'm i don't have full control over my static assets so in this case i chose not to do it that way you definitely could change these urls out to the public the public cdns if you wanted to it would be on a url by url basis so the when we set up cdn support for web jars like this one which i'm not using the web jar cdn this is there's a different it's the js deliver is the cdn for web jars so we did set up a cdn for web drives that has all the web drives all the versions on a public cdn and we did that instead of using the the like jquery public cdn because we can't guarantee a hundred percent compatibility of the web jar what the web dryers have in them and what the public cdns have in them so as a very practical example to this is jquery so you'll see jquery there it doesn't have the version number in the file it has it in the directory name and the way that jquery distributes their files is with the version in the file name not not in a directory name or anything so the cdns all have a version number in the file name which would be okay we could work around that but the problem with it is that jquery because they hard code these the version numbers and the file names they also have to hard code the version numbers actually inside the files because of source maps source maps create a relationship between two different files because they're using the version name in the file name now they have to hard code that version number into the actual file so for web drivers we actually go into the jquery files and we actually rip out the the version numbers out of the files so that the source map links still work but that means that this jquery.min.js is actually not the same file as the one that's on the public cdn so because in web drives we can't guarantee that there's always going to be a correlation between what we distribute through web jars and what the public cdns have we set up our own public cdn which means that we're not going to get as good of caching as if you're using the public cdns but there's going to be trade-offs so yeah okay so that's that's a short deviation on the cdns and that fun stuff um okay so that's how we do it on typesafe.com so gzip now let's go on to mocha so mocha is the testing library and before i go on any other questions about cdns before we go okay so mocha is testing library for javascript that is now part of spt web so to use it we can just go create here a new test and what we do is we create these into an assets directory so in test and then we'll create a new package called assets and then i'm going to create a new file here and these need to be named either ending with spec.js or test.js that's a setting an spt that you can change but that's the the default so i'm going to name this fusespec.js and then let's just copy and paste this stuff here okay so now i've got my test which isn't very exciting i'm just testing that one plus one equals two and now let's go run that test so i'm gonna run activator test that just runs the spt test and i've got some scala tests in there and then i've got the javascript test oh and it's even going to lent the the javascript for me so that's great so it linted it and then you'll see that it ran my javascript test and my one of my scallop tests is now failing so ignore that but here it said that was the test foo should correctly add numbers and it passed so now alongside my scala or java test i see also the output of my javascript test and this is a pretty simple example but you can also use require.js as part of this as well so that if you have if you want to inject basically different dependencies for your tests then in your actual web app you can do that through require.js you can use mocking tools there's some some pretty powerful parts in here that are really just powered through mocha we just use mocha and execute these tests through mocha and report back the results through the standard sbt test runner we um unless there's a rapper if if there's some way to use jasmine from mocha then then no uh i know that it was something that we were looking into uh and so we may be another spt web plug-in or it could be that we could do it through the mocha one i'm not sure there there is a whole different way to do this where you can actually run there's some wrappers in scala so you can actually create scala wrappers around jasmine test using like html unit or selenium webdriver so you could use those as well for jasmine i know people are doing that today but but this actually runs these through the javascript runners which are i think all based on the node api in this case so yeah so i don't know if there's a connection between jasmine and mocha or not but but certainly if there's not we could create an sbt web plugin to do that other questions about spt moca okay all right so that's that's testing um let's go to require.js so with require.js let's go set this up so i'm going to copy some stuff in here and let's go create a new javascript file this will be main.js and there is my my main application so i'm setting up this required configuration you'll see that i'm setting up some some path information for my libraries for my javascript library so i'm saying jquery the path to you is dot lib jquery jquery and that's actually the extracted being extracted from the web jar so this is uh this is now taking the contents of the jquery web jar which i don't think i explicitly have or do i have a dependency on it let's go let's go check out my build.sbt yeah so i don't actually have an explicit dependency on jquery the jquery web jar but it's a transitive dependency of bootstrap so i didn't have to specify the dependency on the jquery web drive but it brought it in as a transitive dependency so what what spt web is doing is it's actually pulling all of the contents of my web jars out into basically an area that becomes available through these paths and we can actually test that out and make sure that it works so if we go to slash assets slash lib slash jquery slash jquery.js then there we see jquery so that's automatically is now there because it was extracted out of the web jar so that's what that path is doing is just saying that when i have a dependency on jquery that's where to get it same thing for bootstrap and then the shim configuration uh exports i'm exporting dollar sign for jquery and i'm setting up a dependency on jquery for bootstrap so that's what the the shim is for this information is already part of the web jars and one of the things that we're trying to figure out next is can we already can we use this information that's already in web jars instead of having to define it here so we haven't figured that piece out yet but hopefully in a future milestone so then in require.js i'm saying all right require bootstrap uh which we'll bring in bootstrap and we'll bring in jquery because bootstrap the bootstrap javascript depends on the jquery javascript and then i'm displaying a modal dialog and now let's go try it out here i'm going to now load in actually that's not how i load it in we have to do this a little bit differently so let me go copy and paste that so with require.js if you're not familiar with it we need to load the require.js library so i'm actually still using the web jar to load in the require.js javascript and then i set a property on my script tag data main and i tell it where to find the main application to run and so this is once j once require.js is loaded then it's going to load up the main.js in this case so that should be good so let's go try it out here and make sure that require.js assets main that should be good where's my modal dialogue no errors gotta love javascript anyone see my problem uh oh yeah that's what i forgot thank you yeah where is that my modal thing it doesn't exist i forgot some information important information from my slides i have that somewhere over here and that's in maine nope index there it is okay let's copy that guy there's my modal all right let's try this again hey there's my modal so when the page loads it's usually now the bootstrap modal so that was that set up all the dependency loading now based on uh required js which you may have noticed that my index page is actually still bringing in um or my main page it's still bringing in jquery manually i should actually be able to now drop that out of there because require.js should be now loading that for me so let's go try that out so it's still working there if we load up the inspect window we should see that that there when we load we load require.js and then we load jquery and then we load bootstrap so so require.js is managing that transitive dependency for me yeah can we get that to optimize that into one cloud yep that's where we're going next so other questions before we go there yeah are you having a little trouble following what's new in the sbt web versus what was in play yeah that was uh specific to play can you just mention something yeah good question so what's what's new and different so the digest plug-in is is new the gzip plug-in is new and then the whole way that we run plugins is new so that's not going to be something that you really see unless you had incompatibility issues between how we were running those asset compilers before and how we're going to run them now so the way that we're running them now is just using the standard the standard javascript asset compilers so there should be no difference in in how you do that and then the upgradability the independent upgradeability of the the actual compilers that's a huge difference as well to upgrade the less compiler the actual less compiler in play before this was pretty much impossible and now it's it's very easy it's just a dependency that you specify so those are the primary differences from from the like what features do i get what new features do i get the the digest and the gzip are are really some of the only real new features but what we're hoping is that this opens up a whole new world of asset compilers that are built on top of this and so we're not going to have all the hack tower stuff that we had before with asset compilers oh yeah so moca the spt moca one that's also totally new we didn't have anything like that before yeah also things like generating uh the source maps that's something that we weren't doing before that now just works out of the box there's all sorts of new parameters so what we try to do for all of these uh is we take all the parameters that the downstream compiler takes and we now move those out into spt settings so all of the compiler parameters that are available on the downstream tools should be available now as spt compile options as well so a whole lot more flexibility in the different parameters that you could specify for the compilers but yeah that's the main stuff yeah cool so any other questions before we do the concatenation on this thing okay before we do concatenation let's let's do something slightly more interesting here so um let we've got our food.js let's tell our main.js here to load also in that foo um so that'll be slash foo and um let's move this thing let's let's leave it there for now actually i think i do it's gonna probably optimize no let's try it without let's make sure we still work okay we still work that's good so now on this project let's run activator stage and this will do now the require uh the require concatenation and minification on that main.js so it'll take a second to run that but then what we should see if we go into target web and somewhere in here i think it's under rjs app and after and then main.js nope that's not the right one somewhere in here it's going to be or not uh let's try this oh you know what i think there's some bug and i have to run a clean before oh actually you know what happened i think it actually compiled over my files i compiled and then it compiled over them so don't run a run while you're doing a stage or else you're going to overwrite your new main.js let's try this one more time here so this is a a problem that i i think we need to address is right now we overwrite main.js based on whether you're doing a run or whether you're doing a stage i think instead we need to generate a different file rather than overwrite them so that's that's a problem with with uh 1.0.0 m2a that i think we need to resolve i just reported that one to the team so hopefully we get that fixed soon but let's see if we get our minified version looks like we're in single threaded javascript lan because you can see maybe you can't see but at the bottom i've pegged one cpu so one cpu is pegged on probably require js running on this thing so this might be a good place to switch over to the native node one rather than the one based on rhino but it's still trying a little slow charge any questions about this one no this is using the rhino trireme one so uh so at this point it's single threaded and it's single threaded running on rhino on the jvm uh we are working on nas horn support as well so nasborne is the new javascript engine in jdk8 which is a whole lot faster than rhino and so when we can run this on nashorn it should be near the same performance as the native node ones but until then it's gonna not be as fast okay great so we're done generating that let's go check out target web when did i say rjs after and then main.js no that's not the right one is it somewhere in here is a main.js there should be a minified one too or not something broken let's look at all these ah there it is i don't know why it's in a weird directory but okay so this is now the the concatenated and um minified version i don't know why it didn't do the min extension but that's a concatenated and minified version of of that app so we should see in there the contents of foo unless it optimized it out which is which is possible but um alert asdf do we see an alert asdf in there it may have optimized that thing out oh there it is thank you sweet so um yeah so there there it did concatenate the files together and and minify so uh the other thing that you'll see in there is that the paths so the by default what this what the require.js plugin is doing when it creates the concatenated minified version it's also changing my paths and you'll see that the new path for jquery is now using the public cdn for jquery with the with the versioned url for that file in it and there's some parameter where if we wanted to take jquery.js and take uh bootstrap.js and we wanted to merge those into one concatenated file i think there's a parameter to do that and require js but in i think the default well obviously the default is to not do that with these external pieces and so instead it changes the path to be the the public cdn path for those which isn't the the the public cdn it's the web jars public cdn for those files but they are versioned in the url so that means you can do the far future expires on these well the cdn does that and so you should never have to request these again there we go so that's the required js plugin questions about that okay let's i think that that's it so if you want to learn more about this stuff all the projects are on github under the spt organization so spt web is kind of the main entry point and then there's different projects for all the different plugins but i think all of them are linked to from the spt web project and then the play two three migration guide which is available on playframework.com and then if we go to the docs and then select uh two three um i think we want two three snapshot there should be a migration guide here and so you can learn more a lot more about the spt web stuff and how it relates to play in in the the migration guide there as well so that's spt web so any questions before we wrap up and go get a beer or something are we getting a beer alexi is that the plan yeah yes okay perfect sounds good okay any last questions about anything yeah like who would you say is the uh kind of like the most high scale um you know web service that is based on play specifically based on play um good question so uh clout is a good example clout does i think it was 10 billion transactions a day through the through like eight play servers or something um so that's a good one linkedin has been in the process of basically standardizing on play they're in the process of changing over their homepage to play um so that'll definitely be a big one um walmart canada uh is now a play application i don't i'm sure they get a ton of traffic they're not using play as far as i know they're they're using akka and scala but haven't used play as far as i know yeah um yeah so those are the examples that come to mind maybe others know of other ones but yeah any other yeah go ahead so part of the part of the migration point is that you can use sbt web outside of play yeah what do you what do you lose if anything you use it in some other web framework nothing yeah yeah as uh it generates resources into a classes directory and most web most java web frameworks serve static assets out of the class path so that works really well you could you don't have to do it that way but because they're already going into a managed classes directory for sbt then super easy like play does to serve assets out of the class path so yeah i know that people are already using spray with sbt web so and i'm sure you could use finagle and could use um if you can use the right version of spt with finagle and you could use scilantra and whatever so i'm sure that all of them would would work with this as long as the sbt version compatibility works yeah yeah cool and if you have any feedback on this stuff uh post on the the play framework list or email me or whatever