Scale By The Bay 2019: Jason Swartz, High Performance Serverless Functions in Scala

Scale By The Bay 2019: Jason Swartz, High Performance Serverless Functions in Scala

Recording: Scale By The Bay 2019: Jason Swartz, High Performance Serverless Functions in Scala

[Music] today's talk is gonna be about high-performance surplus functions with Scala it's great to be here it's scaled by the bay so hello scale by the bay wait a sec no that's San Francisco yeah hello scale by the way in Oakland thanks for coming to my hometown makes it easier for me this works out really nicely it's a beautiful venue in it yeah really liking it here alright so this is gonna be talked about serverless functions and Scala specifically whoops just like pressing that specifically about AWS lambda so I want focus on AWS I know we have cool cloud and azure and other ways to do service this is going to be AWS specific today all right so November 14th we're all set well to start off I kind of like to tell tales when I give technical talks at developer conferences in Masonic temples in Oakland usually I like to talk about one of my favorite tales vanilla Sally and the growling lambs but you probably already know it right who knows this story No anyway all right I'm gonna go over to the other side anyone here know this story vanilla Sally growling oh well you know story vanilla Sally growling Lambs no I mean really it's not that it's really about using vanilla Scala and compiling it to native code with growl VM and running it on AWS lambda that's what the real takeaway here vanilla Scala growl VM lambda I just thought I'd make it kind of fun you know vanilla selling growling loves no vanilla Scala and growl and lambda hopefully you either walk away from this remembering vanilla Scala and growl VM worked great with AWS lambda or you'll be thinking what the heck was that stupid fairy tale that guy kept telling us it makes no sense so vanilla Scala works great if you compile it to growl VM and run on lambda you get high-performance service here's a here's a breakdown of how this works so first you write your lambda functions in regular Scala use built on SBT you know set it all up have these nice endpoints then you go ahead and package it up with something like SBT assembly which builds all of your chart dependencies into one big jar file and then you get grah vm and you run the native image you run the native image app and it compiles it to native Linux executable which is a great part of girl vm now you don't run this in the JVM you actually have like a Linux you have Linux code then you could use something like serverless or Sam and set up an AWS lambda function have it run behind API Gateway may be fed by events from sqs and you're all set make sure to give it lots of memory this works much better with memory and then deploy it these are the five simple steps for server lists high performance apps written in Scala guaranteed works every time so oh I just gave it away oh okay so if you can remember this part vanilla Scala and I know I'm giving it away really early in the talk and that's pretty crazy and all but then you'll be able to write high-performance service functions so it's good thanks everyone let's find out if there's more cookies left now okay there's actually you're supposed to start with the problem and then go to the solution I realized that normal talks you're supposed to talk about the problem right I mean and then go to the solution okay let me try this again here's the problem I kind of want to write a slack up who's a slack here a lot of people yeah and who doesn't want to write to you know a fun slack up specifically I want to write in Scala cuz Scala's kind of fun right it's kind of a great language to write it in so I want to have a slack app I want to write in oh yeah because I'm doing it myself I don't want to spend a lot of money so I don't want to have this in ec2 or some other dedicated service I want to use something like lambda so that you know it can scale way up if there's lots of traffic and it can scale way down to zero and not cost me anything if it's not running so I want to slack up and well write in Scala and I don't want to spend a lot of money that's kind of my problem statement there for the whole talk I went ahead and chose running Scala on AWS lambda and praised another with AWS I think it's pretty cost efficient you can actually stay on the free tier most of the time if you're just using lambda which works out really great I also really like the server list framework it lets you it's cross language it lets you set up a llamó file with your whole configuration you say here's my endpoints it does all the conformation it sets up the the stack it sets up the API gateway can have sqs events sent in and trigger your lambda it works really great this for me is like the perfect setup for writing service apps in Scala okay it's kind of a twitch thing it mean it seems good all right so this seems good to me this works pretty well hard part here is if you write a slack app you're supposed to return response to a user interaction in less than three seconds someone clicks on something in slack they click on a button or they click on the link or something it goes out to your app you have to do everything and then trigger the response within three seconds so you have to you have to get that back but even more important than three seconds according to Jacob Nielsen if users click on something it takes more than a second for something to come back and show an interaction they kind of lose the user flow a good user flow you have to have something snappy and responsive in less than a second even more so if you want to make it look like it's instantaneous like I clicked and something happen you have to have a response time about 100 milliseconds or less so you really have to make this snappy so if you're gonna if you're gonna have a service function which is going to drive a UI like a slack bot so or a slight interactive slack app you have to have really fast response times to make sure users are going to want to use this so you're right Chris okay so where's this year issue here is that JVM when you're lambda is kind of a slow solution and it doesn't really meet the needs for getting this into let alone getting this in 100 milliseconds it doesn't always meet the needs for having a snappy user flow in less than a second they may not even meet the needs for returning less than three seconds it's just it's kind of slow cool core thing is someone's calling your lambda function over and over again the response is pretty good what happens when no one calls your lambda function for a few minutes does anyone know what happens then yeah it shuts shuts down the JVM so someone has a call call coming in the endpoint comes in lambda will have to start up the JVM that takes extra time that's known as the cold boot and it's chilly yeah it no it's it's really slow and this is a huge problem for any of the jvm languages to run them in service I did some testing with different sizes with a pretty basic service app that does a little computation mostly returns one thing that really works well is memory if you apparently if you throw a memory at it or if you throw money at it and you run with more memory up to the limit of three gigs which is the max right now aw is lambda the cold response times really goes down a lot but still not really that great if I layer on top of this the slack time for responses and realize you know even if I have like say two gigs of memory I still want to do things my slack up we need to call other api's maybe call stripe maybe call slack api's those take half a second it's pretty unlikely I'm going to have in any every case returned in under three seconds let alone try to have a good user flow where people stay in the flow and do things it's kind of impossible with just pure VM with a pure JVM all right which brings us back to the tale of growling lands yeah all right so who hasn't heard of growl VM oh all right so this is no Oracle project it's it's been a full release it's an alternate VM for running a number of different languages pi and Ruby and many others but it's also a way to take Java apps and compile that compel them to native code and run them with some restrictions in such a way that you get a really fast execution time this is taken from the growl vm website itself and it's true if you create a native image with this you get really fast startup times here's an example taken from their site you can see that I have a 90% in price a 90% of the time by combined taking this bytecode compiling it to a linux executable which then includes the growl runtime VM itself you got much better start of times you kind of kill that whole JVM startup time issue that we see a lot alright so AWS lambda you know it works with like Python and go and JavaScript and Java and Scala calls entry points works a little differently when you use Linux executables you can use Linux executables in AWS it has something called the custom runtime environment where they have a base Amazon Linux based Amazon Linux it does have Java a few other things but you're supposed to have a Linux executable you compile it it starts it up you do an HTTP gets to say hey local server what's the next request and it returned every som one calls API gateway to get to your function it then returns this whole JSON which says ok here's the whole API gateway here's the query parameters here's the headers here's everything that comes in same thing if you have sqs events which trigger your lambda you have to have basically an infinite loop that just calls this local thing gets the response if you have a response you have to then write it back to the same URL and then call it again it's a little bit funny but it does work I mean you could use pretty much any language this is what I use with growl VM I have a native Linux executable and I just call this over and over again I am using an older version of the growl VM I do the compilation in crawl in docker because AWS lambda our custom runtime is Linux and I'm using a Mac so if you're on a Mac you and you want to run this in Linux especially in AWS custom time you pretty much have to cross compile this in a docker file but this works really well for me they are oracle has a number of them available here I'm saying yes I wanted to do native image here's my big fat jar file that I built I do want to do certain types of networking HTTP and HTTPS and it sets up this image for me the other thing I have is my core build script I'm using SBT assembly to say okay going and test it out and then we run assembly build all the dependencies and transitive dependencies get put into one big jar file and then run the docker and get the output and then I zip up not just the output the links executable but you can see I'm including CA certs and Lib sunic these are two JVM files that come with the Linux JVM that make it possible to do HTTP this makes it possible to do the security necessary for supporting certificates so I can have an HTTP client that goes out and works properly so this is kind of a hard step but you have to do that if you want to have a Linux executable running a native in AWS lambda in the custom runtime with from a Java program fortunately when you then run this the response response times are pretty great you can see that cold start times and I'm not sure why the 256 one I kind of want run again to measure it but all the response times are under 250 milliseconds pretty much for cold starts warm starts even better and to be honest when I run this I usually have response times of about 50 ml 50 to 80 milliseconds someone clicks something in my slack app it calls back to me I do some things I write back completely finish even doing a blocking right back to slack usually takes about 80 milliseconds total AWS then charges me for 100 milliseconds of it very cheap it stays on the free tier very fast turnaround time I mean this works great if you look at that one second for user flow we are way beneath there it means that someone clicks and calls our app we can go out to a database we can go out to a queue we can maybe call some remote API so we have a lot of room to work with so we can be really fast and have nice efficient applications that have wonderful user interface so grow VM and the aw slam to work together really well it's it turns out to be a wonderful match so enough of the growling lamb goodbye let's talk about vanilla Sally or you know specifically vanilla so skaaland growl VM though this works with Scala you can build jar files you can then create a fat jar file issue is that there's a lot of problems getting this to work this doesn't work super well it's actually a little challenging to get your Scala apps compiled to Linux code with growl VM plain Scala code works out well case classes and options and most things you do work do work pretty well but there's no reflection allowed one thing they remove in grow vm is you can't do any runtime reflection at all which actually means there are some there's some pattern matching that doesn't quite work in Scala if it requires reflection to reflect based on the class type so you're not gonna be able to check the type of things J&I does work though if you're packaging for Linux you have to remember to use Linux libraries and link with them instead of Mac libraries because it depends on you know the runtime environment but loading classes dynamically doesn't work at all the crawl VM wants to have everything set up and initialized at Build time not at runtime runtime you're not allowed to load classes what this kind of means is that you can use some pure Java libraries like Apache HTTP components from I think like I think it's 20 years old but it it works great it doesn't do any class loading I would love to use Scala HTTP think HPS things like akka HTTP and such but they all do some type of either reflection or dynamic class loading and they just won't work in raw JSON Eider is a great Jason solution it works really well extremely simple you can choose to use it with implicit or not but it doesn't do reflection it's great if I believe most of the other J's scholar Jason libraries I tried did do sometimes a reflection and they didn't work in grove IAM so this kind of gets back to the idea of well you kind of have to do things yourself uses a few libraries as possible and written API clients if you're calling a rest api write the code around apache components you know set up your own options used try use all the scholar things you like but if you do it yourself it works great if you use things off the if you use other people's API clients I think I tried stripe and a few other different API clients I couldn't get them to work they all did some type of reflection or quite dynamic class loading underneath for AWS I did need to get AWS on SDK for Java working so you do things I call it dynamo I did find there are ways in call VM to say okay there's something which might intensive dynamic initialization I'm going to call it up by name this is an internal JVM classes used indirectly by AWS and to make sure you grow VM doesn't have errors when I build it you have to call some things out directly so a little bit hard to deal with core thing to remember though is no not this but vanilla Scala but if you have plain vanilla Scala and you compile that with growl VM to Lynx executable that works really well so that's that's what I want you to remember more than the silly fairy tales or the really beautiful typography of this presentation is this part this is the this is the key to success for high-performance service applications at least with AWS lambda so I think those are is actually pretty good where I'm at where am I'm a near the end oh oh I forgot to introduce myself ah you're supposed to do this at the beginning right all right all right so hey guys okay another twitch thing so I'm Jason Schwartz and I like to work with Scala and I've worked with a number of places and I like working with technology and engineering and on that twitch right now wearing the twitch logo thing we don't really do Scala and they don't let me code anyways so it's kind of fun to do Scala things on the side it's a really fun language to work with I've been using Scala for about I guess seven eight years and it's been great I've worked with alexei on this scala by the bay NSF Scala and it's really fun so it's great to come here and talk about this and share what I learned here I'd be happy to talk about Scala or any of these companies or or being at twitch or anything like that or video games so please come find me and I'm open for questions now [Applause] I've been are there messages when something is going to be dynamically loaded what do you know and fortunately I found all the issues while running this locally and docker as a test you can certainly since it's just a Linux executable I would just have a dot separate docker file to then run the Linux executables see if it can start up and call a local HTTP URL and I'd find it pretty quickly a lot of the cases you find those at compile time growl VM is really good at looking through and saying okay this is usually reflection or this is going to be dynamically loaded but there were a few cases were actually then had to run it locally in docker to actually see it fail and girls say well I can't run you're trying to do something dynamically I showed reflect config dot Jason there's a number of different ways that you can say hey take this thing which is trying to do runtime initialization and do build time initialization so it pre initializes it and then gets it ready but I found is really hard to get that working for most things I tried with akka and H I think Circe and I tried it with akka heb and all that all the really cool libraries I couldn't get to work yes I haven't I liked grog because it was new and shiny and really good Docs and the native image the fact that they had the docker Dockers ready with the type of Linux made the cross-compilation super easy yes [Music] Agro yeah are not yet I think Argonaut does some work with I think I've seen articles on Argonaut and crawl yeah girls great it's just it's a it's really interesting how they said okay if we don't do any reflection or you know runtime type introspection it turns out we can actually run this as in it as a native executable any questions from this this side of the room their questions were that good that's all right oh yes sir I would compile it in initialize it at runtime when the app first starts up and build it into links executable and try it out crawl is pretty good at checking and verifying that there's going to be some type of runtime initialization for example use of the JVM things like class dot for instance or other type of Jerry for runtime introspection so it does pretty good job searching through those and it's about a two minute compiled from from so it can take some time to do it but I would just say try it the hard part is yeah in many cases you off to get the most vanilla basic solution yes no what does cloud state [Music] pretty anything very young nice is anyone talking about that at the conference cool how much more time for questions do we have to two minutes or two questions yes you mean decrease the response time well the let's see if I'm still let me go back to my measurements of actually renting these things throwing more memory at it works pretty well if you rent with three gigabytes at dusk AWS does charge more for this but for some reason I'm not sure if it's cached or it just keeps it running but the cold start time is it's over a second but it works pretty well it's possible that if you go to Google cloud or you go to Azure I don't know if these if they have the exact same issues some of this might be dependent on the implementation of AWS lambda we have time for one more question no one wants to know about the silly fairy tale yes and back a collection of libraries that you have tested that you have available online somewhere it's just Apache HTTP components and Jason eiders Jason eiders Scala and that's it I have a very clean happy build SPT I couldn't get anything I know that an AWS SDK so Jason Hydra Scala Apache components and the AWS SDK though I did have to do some Jason configuration I'm not using anything else I just found making my own Scala type a HTTP wrapper with tries with my own response body within a fast Jason back and forth it works so well I love you know having tries or having returning in either of either a you know message or an error object and handling pattern-matching I was pretty I turned out to be pretty happy just using you know these very based components all right well thanks everyone I'd be happy to talk about any of these subjects in the future [Music]