SBTB 2019: James Ward & Josh Suereth, Serverless Scala - Functions as SuperDuperMicroServices
[Music] welcome every will welcome everyone this is surrealist Scala functions as super duper Micra sir Oh micro-service is yes we came up with a name can you tell yep you know that was all ours all ours I'm James Ward I'm a developer advocate at Google I'm Josh thread I'm a staff at our staff software engineer at Google yes okay so we're gonna talk about first about serverless what it is well what we think it is and then we're gonna dive into some code and talk about architecture and all that kind of good stuff so first serverless what is serverless so serverless the way I like to think about it is that it's pay for what you used use managed hosting so it's managed for you you shouldn't have to manage the stuff like you just deploy it and runs and you don't have to think about the operations and all that kind of stuff and then it's paid for what you use which means ultimately means we're trying to match the demand on our application with the supply so you could imagine that if you left a bunch of servers running and they weren't being used then either your cost would have to be really high to account for that or you would just want to shut down those things when they're not being used and so this is I think a core part of serverless is being able to turn things down when they're not being used easily and having your applications account for that ability so that they can be turned off when they're not being used remember you thinking like what if I'm not like using a cloud provider I've got some on-prem stuff and I am not paying for things like we paid for our servers long ago and so I'm not actually paying for these things what does it mean in that that world and what I look at that as is that Netflix calls something a trough so they have this whole thing where when you're not using your resources in their infrastructure you can return those resources to the trough and then other jobs that aren't like user critical don't have like the expectation of time like how long it's going to take to run them they can pull resources out of the trough so this is like machine learning training stuff can be pulled out of the trough and used that way so so even if you're on Prem you've bought your servers I think this still works because you still may want to return those resources to someplace that they can be used for other things when you're not using them yeah it's recycling your head room recycling your head room that's right so you can imagine like you know you've paid for all these servers if you're on Prem and you want to return them to the trough it's actually a great way to make money on Bitcoin so all of you all should be able to get rich by all the slack capacity in your your on-prem servers right if you go serverless and send me a little cut please I'm going to use that okay there's a few different flavors of server lists that have that have come out there's function as a service and so this is where you deploy a function and it goes into this managed mode and then you know it'll spin up the the resources required to run that function when requests come in or when you need to process some job or whatever it may be but it's at the the granularities at the function level and then there's platform-as-a-service which is really about deploying applications and platform as-a-service can also be serverless you're just shutting down a whole application instead of just a little function okay so that's our quick little kind of thoughts on server lists and we're gonna do more thoughts on server lists in the panel tonight so you should come to that that's gonna be what we're our thoughts on surveillance again pay for what you use o and managed managed that's right yeah why were those highlighted after the fact after we just walked through them to reinforce the point that's right right yeah I mean it was a bug in our slides I'm supposed to highlight those we don't want you to remember anything from the slides yeah that's good we should talk about cloud right so for our demo later we're gonna be using something called cloud run which is service for containers so it's not function as a service per se it's anything that you can run in a container so I have a little sample where I run netcat in a container it's like the the lightest weight little webserver you've ever seen it's pretty awesome so cloud running this we're going to be using but cloud run is based on an open source project called K native and we just did a workshop yesterday on that if you're there it was awesome well I thought it was awesome because everything worked Josh was there you were there too yeah you know so Kenny we did it all on kan native and soaked a native if you're looking for something serverless on top of kubernetes that's k native and yeah ok so oftentimes when you are learning about serverless you are using serverless you run into something called cold starts and this is like the most like contentious part of serverless so you're gonna get cold starts the problem with what cold starts are is when you cuz you have to shut these things down when you're not using them what happens when you need to use them you need to start them back up right and so this takes time to start up things and and so when that that time that's being taken is called a cold start and so we wanted to talk a little bit about some strategies for dealing with cold starts so it's because we're scaling scissor actually it's not my slide is what's wrong with these slides it's not just about scaling to zero because what if you are at one and you need to go to two then you get a cold start yeah so the way the way to think about cold starts is more about you're trading money for your 99th percentile latency right do I want more money or do I want a better 99th percentile latency 99th percentile agency would be 99 percent of my requests are served in a certain amount of time everyone smile and have to boot up a new machine right that request is gonna get really slow possibly embarrassingly slow depending on the service so not every service you can trade you know that latency for money but sometimes you can yeah so for example if you're building like a voice system that's supposed to like respond it would be quickly it would be really weird if it didn't respond immediately that would just be a weird experience or if like somebody's like checking out of a shopping cart and it takes for a while takes a while they may just like go and do something else and cancel their order and you don't want that so some jobs are very sensitive to cold starts and then other things like ETL jobs not incentive to it so sometimes it's an issue sometimes it's not so one of the ways that you can deal with cold starts is just try to reduce your startup time that it takes to go from from nothing to something and there are some strategies for how to do that if you're in the node world they're like Oh keep your like node application under five megabytes and then like it magically will start up quickly every single time and then you're like what about database connection pools and they're like well we're server list so we don't have any database connection pools then you're like okay well that's gonna be take a long time to connect to the database on every request anyways so so it's sometimes you can reduce your startup time and deal with some of the problems but then what if you have to like warm up a cache on on your system that's something that is you just can't deal with that sometimes you just need warm stuff and so so yeah that can be hard if you're on the JVM if you're programming on the JVM expect most of your Scala developers here then you may be able to use Gras VM which is in ahead of time compiler which compiles down to a native binary which allows you to start up much faster and so this is a great option that's actually what we used for the demo that we're gonna show you in a little bit is grow VM with wizzy oh yeah and one of the important things about grow VM is your training consistent latency for potentially optimal throughput right so JVM servers meant to sit there a long time take a long time to JIT and then they get to their efficient world and it takes a while to get there with grow vm you get this nice consistent startup time and a consistent serving path right but you might not get the same optimal performance so again there's a trade-off and in this case it fits really well with that same server illustrate off right I get that consistency I need to know how to scale so I use SBT a lot you you helped with that project and I I don't use the console on SBT very much so we usually want to use SBT I'm getting a cold start on SBT right every time yeah but somebody has taken SBT and run it through crawl VM and produced an SPT binary that starts up just instantly Wow yeah no more cold starts even for your dev tools oh man amazing you know the the whole SBT server thing by the way was to avoid cold starts ever on your machine most of the development and SBT for like the past you know three years or so it was to like keep it persistent always and just sucking up all the RAM on your machine so that your fan would turn on and then also make sure that it was fast yeah just trade-offs it's all about trade-offs okay so transitioning out of server list we're gonna talk about building server apps let's talk about the way that I've built a lot of Scala server apps alright is with Play Framework where it's so nice because you just like have a function that takes that has a request as a parameter and returns a response like like so simple I love it I love it but what's what what what do you have to pass into there yeah what no see you don't there's nothing there oh it oh okay there's nothing else that's right there's a lot of stuff lurking under the covers that you actually need in order to be able to you take a request and return a response yeah so this is where we're talking about dealing with you know how to do threading what kind of how were we pull our configuration from how to serialize things it uses actors under the covers so how we instantiate actors and push them around right there's a lotta law a lot of context in that little bit of code that's hidden from you you don't see it but it's there and so when I need to test this yeah when you need to test this man gets a little tricky yeah it's a little tricky because then you have to recreate all those things that we're providing the information to be able to handle that request and plays like tried to make that a little bit better with this with application thing and but it takes a while to like run your test because of this and all sorts of stuff so yeah you you might be booting up a little mini play application every single time you want a unit test a little tiny bit of your code right yeah that's a lot it's a lot going on so the question for us is we've built a lot of server apps and and try to think about okay how do we like go through the different cycles of a server app from dev to test to prod like is there a better way is there a better model to be able to move those environments without having to have all this kind of stuff needed around my my actual logic that I'm trying to test yeah specifically that function right you know it's it's not going to run on just a web server in production and then your unit test right you're probably gonna do a little bit of testing across components especially with micro services you have a lot of these services and like one thing we do is I don't know if I corn this term so I don't know if this is a real term but I call it edge testing right you test your micro service to its dependency micro services you start those up you make sure that they can communicate together but you stub everything else out right and you do this because you need an integration test but you know you might have hundreds of micro services you don't want to stand up all hundred micro services or run your test in production because of some instability issues or you know possibly corruption of data and all that kind of stuff so instead you'd really like to try to do some edge testing here between the things and so you have all of these different environments that you're taking this code and pushing it into and you're not gonna have one config file for like the production config you're gonna have production you're gonna have integration testing you might have like a QA environment you know all sorts of things that you have to put this code into and they're all subtly different environments yeah it's tricky it's tricky yeah so we want to move from write once run anywhere to write once run everywhere right all of these environments I need to not just run on one at a time I need to run on all of them and adapt to them yeah yeah so how do we do that well I think we need to focus our logic right like when I'm in development what do I care about just the logic the business yeah yeah the business domain right but when I'm choice and production nation J like all I care about is moving JSON from A to B that's what I care about yeah and then in production what I care about how fast to the JSON is moving how quickly can I get that Jason in and out and then for testing you know is the JSON the right shape yeah you know does it look like a bunny does it look like a turtle I don't know we don't know but but I need to know that's why we yeah yep okay so we we wanted to experiment with some of these ideas around this and so we built an application as an example that is a chatbot not like the slack kind well maybe actually but we were like okay can we like have something that we can just like test the actual logic on just for like local development uses and then can we expose that same logic as different protocols let's do standard and standard out why not do tell Matt because and then hey let's write a web server from scratch because that would be fun too yeah well I mean once you start with telnet you just you go all the way but gtp is just a little layer on top tell me it's true but yeah telling it's really easy to test on your system right I just tell that into a server and bam boom right I just send text good yeah easy okay so we built this simple little chat application which we'll see in a little bit so we used a significant indentation no significant whitespace in Scala too and it was amazing is that a beautiful cut yeah we like it for horizontal significant whitespace horizontal significant whitespace yes a new thing but this is what the code should look like this is what we want to focus on for business logic right like after the user says something and we've kind of recognized what it is we have something to kind of interpret it they've told us a language and if they tell us a language then we're gonna say whether or not you know they set the right language and if they haven't given us a language that they say anything else we don't care what it is we'll just say hey what's the best language and you know there's all there's only one right answer and for this app yeah because it's what it's written in that's right so this is like the logic that we want to write this is the logic that we want to test this is like as close as we can get but yeah we we can't do that because there's a lot of other things in here that kind of get in the way right there's logging there's monitoring there's all sorts of little components you know that we're going to be adding to this code and it turns out not to look that simple as we add in all those things that's right so enter the interpreter pattern yeah so what we're gonna do is we're gonna find a way to write that logic and kind of hide all the other stuff so that our business logic looks as close to that ideal as possible and we can test that business logic but we can also put in the little bits that we need that are kind of more horizontal so here what we're doing is we're constructing a data type called talk where we have different operations that you can do when you talk right so I can say something I can ask for something or I can output several different things I could say and ask for things okay so that's those are the operations that I have and now I in x-ray I'm returning this like operation that says what you should do but I don't actually do it right so before we were actually calling a method that would do that say do the talk now we're returning something that just you know it's just a piece of data right it doesn't actually do the behavior and so all of those little pieces of logic around when I say something I want a log you know how many times I said this thing or I want to log how many times the user said something right all that's hidden and somewhere else I it's not in this logic I can test it without having to deal with all that extra complexity so we can take this meta program that is just our like core logic and the the data of that and then we can wrap around an interpreter and the environment that then is needed so in development we're gonna have a different environment and testing we're gonna have a different one in production we'll have different ones so on and so forth yeah those of you who've been in Scala for any number of years you've probably heard this message preached over and over and over and reinvented about a bajillion times in the functional community right so these are these are our projections of what might happen given this timeline but these are the different ways that this idea has shown up in FP we've been just reinventing the interpreter pattern in many many many different ways so we should have made a prediction there we should have made a prediction on when we're actually gonna figure it out yeah it's true because we just said question mark question mark well colonize Mars already live on Mars that's true you do yeah Mars Pennsylvania it's a place anyway okay so we used zo4 for our effect interpreter and so let's just talk a little bit about what Zeo is and before we get into a little bit of the code sure so Zeo is there's two ways to describe it right there's a glib way that you use when you're trying to be coy right and then there's the actual like how you describe it so I'll give you the glib way first because I just can't help myself but you just say by functor i/o and then you'd walk away right but no whatever what it really is is just to drop the mic and walk away by functor aiya okay all right so what it really is is you have you're trying to encapsulate your program and you're remembering three things in your program so you're building up these values that are programs that can execute and the three things you remember are what's the environment I need to run this program like what are the components that I need to run it do I need a logger do I need some kind of monitoring do I need access to threading like what's in that environment that I need that's the first type the second type is what errors are am I going to return it actually makes you kind of remember this it doesn't just make everything throwable you can pick whatever you want and the last thing is what will this program return when it executes okay so that that thing where we made that you know abstract syntax tree what CIO gives you is a building block to construct more and with these three components to it so you can kind of make applications this way and then you essentially can just build a chain of zeros that becomes your full program but at any point you can take part of those Zio's out and test them independently or provide a different environment to them so here's an example where he abstract out that talk ADT that we had before so here we have a talk service and in Zeo the way you make an environment is you're going to make something that with you know like like trait extension methods into an environment so you kind of bundle these things together it has a name and that name used to terrify Scala developers so we never used it but it is called dependency injection I thought the reader monad oh sorry but okay okay it's also the reader monad that's one of the functors in the by functor any so it's called dependency injection and the this is this is we're defining a trait which which is saying hey in that environment type you need to provide this service right if if you require this type and then when you define the service there's a bit of boilerplate here it's probably like the grossest part of zeo honestly the boilerplate but it it tends to work out so you'd find your service you find your methods on it you know they return these Zeo instances that you can then compose with everything and what's interesting is now I can define an application that uses my talk service and I can take in the type signatures I can know when I'm using the the talk service and I can provide a talk service to my Xena program and remove the need for it later so I can basically compose applications where I can decide when I need to instantiate this thing and how it's used and I can kind of peel on and off my dependencies it's pretty powerful yeah so that's what we built was our talk service and that's not exactly what it looked like in code but pretty close right then with this significant horizontal and program all the time yeah you never get horizontal space on the right side oh right sorry slide yeah we'll fix that but this one doesn't compile but you can see how this is this is a lot closer to what we really wanted our business logic to look like right and that that's the goal is to make sure that business logic matches the way you think when you're doing business logic it should not have all of those other things blood-in right yep boom the emphasis here is this program relies on that talk service being provided right so I can compose 0 here but the environment is remembered so I have a dependency on the talk service for any of this code to be implemented so if I'm going to test it if I'm going to run it I need to provide a talk service it remembers my dependencies so I have dependency injection but the dependency injection is actually type safe it's remembered throughout the system and that's actually important right if I start using some other zio services like blocking threads that kind of thing they will show up in that type signature and it will be remembered so it's a really interesting concept to dependency injection so then we need to provide an environment to actually run this thing so we as I mentioned we had different protocols that we could use and so ultimately different environments so we have a cloud environment that is able to execute the the request over HTTP then we have the console one which is just just standard out and we're providing different environments into this thing so that it operates differently the business logic doesn't change just the environment that changes around it what's interesting is after you provide the environment you notice the environment becomes any because I can run anywhere now like I provided all the necessary services for this thing to execute so it can just run just run and you now have environmental cake anyone anyone remember the cake pattern in Scala yes yeah also also a bad word yeah but it's a you know that's so this is dependency injection with kind of a cake pattern you bake this environment together of all the things that are necessary to run that program and you very clearly and explicitly denote them in your types nice kick okay so if we want to add like monitoring into our into our business logic then we can we can just have something that has monitoring in it and we can change that monitoring environment what is actually doing so if we're running in production we're going to send that up to our monitoring system for running locally we can just make it do nothing if we want okay so what does all this have to do with serverless so the way that that we wanted to the idea we wanted to experiment with is that we look at zo and and by functor io and all that stuff is allowing us to abstract from the environment that we're running on and then serverless allows us to abstract from the operations I think I think that's backwards with serverless is abstracting from the environment oh no you're right you're right right but see there's a cloud on that one yeah my bed yeah and now I see little boxes it's like the the layered environment yeah the layer at different places you run it's right yeah and then we get this beautiful cake in the cloud okay so that was that was the idea we talked a little bit about that being able to you change change the implementation change the protocols across the different environments yeah so the the the demo we're gonna show you some of the different environments that this code that you were looking at actually runs on and another another interesting thing is it's actually the same binary we put a little thing that detects what environment it's in and then instantiates the program correctly provides the appropriate resources right and that's that's a nice little if-else statement that you can actually read and understand what's going on as opposed to trying to track through xml or annotations and figure out which one is coming through right it's it's it's more clear concise code keep talking we're getting a cold start now so we're gonna run the console version of this demo and and then we're gonna show you the the Google home one as well okay survey says what's the best programming language Java is that what I heard yeah I think I heard that wrong oh man okay Scala is the best programming language correct all right there's our console version yeah okay now let's let's try this one now I'm gonna hold my microphone up so you yes okay is it on okay Google Talk see ya talk to scholars yeah sure I like talking to you know okay google talk to Scala Zia getting the test version of Scala zio survey says what is the best programming language rust wrong survey says what is the best programming language Scala say that one more time of course cancel that's a little Easter egg for you yep nice okay all the code is up on my github if you want to check it out we have one minute left any questions any questions in the last minute well no so the the code is the thing so the question is what would the worst programming language be the codes the thing answering the question and it's written in Scala so of course it thinks itself is the best right like it's the code during the end of narcissus we didn't decide that that was the code yeah okay one more question [Music] that's right yeah so um you cannot use things that use reflection there is a way around it you have to like create a file that talks about the reflection and all that kind of stuff but in general yes because we're using it's called a zero and not using anything that uses reflection we didn't have to worry about that if you look in this repo there's the docker file which builds the growl VM native image so if you're interested in how we actually built it then check it out but definitely one of the interesting things about using libraries that only using libraries that don't use reflection as that they work well in grub yeah yep good question okay we're out of time but hopefully that was useful and fun thanks so much thank you [Applause] [Music]