SBTB 2014, Phil Monroe: API Development with Scala and Dropwizard

Recording: SBTB 2014, Phil Monroe: API Development with Scala and Dropwizard

uh hi my name is phil and i'm a scholar developer all right cool um where are you going stop all right cool so i'm going to be talking about a drop wizard on top of scala so a little bit about me my name is phil monroe um senior software engineer at workday i've been doing skal about a year and a half now um been enjoying it so um i'm going to go through things probably pretty quick it's a lightning talk so if you want to look at the slides filming dot com slides drop wizard example application on github demo on heroku cool so what is drop wizard it is basically everything you wanted when you were developing with java put together for you and you know made into a library that's just convention over configuration so you don't have to worry about you know am i building a web container am i doing this am i doing that you just have a simple application and simple configuration and the thing starts for you so so just to start off what is it like running a drop wizard application it packages up all into one jar you just say java jar first arguments server starting server second argument's config file generally just simple yaml or possibly json if you want if you're super ninja you can actually you know redefine your main method get rid of those two arguments and your operations guys will love you so all they have to do is just you know curl a jar start it up and you're good to go cool so this is what it actually looks like can't really see it too well but you know it gives you a nice little banner when things start up you get a listing of all of your endpoints and basically just you know looks nice so let's look at just a simple application um there's very little that you need to do there's just two methods that you need to implement initialize where this is just you have your you know your setup code you can see here i've got you know what i'm calling a bundle these are basically just initializers i'll talk a little bit more about them later but you just initialize some things and then run this is where you actually register hey jersey i'm you know i'm starting up with you know with this context well i guess one thing i forgot to mention so drop wizard uses you know jetty under the hood um uses jax rs to actually set up its endpoints so you just register with jersey hey i'm registering these things so when you start up you're good to go um config files they're simple um you just you know they're just basically plain old java objects you can use case classes and it automatically just serializes these yaml files into them for you and if you want to use heroku and follow the the 12 factor app kind of mentality you can just get system properties and you're good to go one thing that's nice is it also uses validations for you so you can actually say you know i have to have you know i can't just get null from from the environment or or else you know everything will fail for you so that leaves your config files looking to be nice simple readable yaml files um easy to work with again makes your ops guys very happy to not have to deal with mile long java properties files um so resources these are the things that actually um you know connect your uh these are the actual like endpoints that you define and work with um so here's just a very simple hello world um you can see it's just it's just a class uh it produces you know text plane um and i've just got one method on it that's a get and as you can see i just return a string hello world um you know very simple it's a little bit different from from play or some of the other frameworks where you define your routes one place and they link up to a class another place with this you know you have everything all kind of consolidated in one little class that's easily testable and you can clearly say hey what's going on and then yeah so once you have a resource you just register it with the runtime and and you're good to go um cool so here's a little bit more complicated example it's actually still really stupid simple but you know if we're actually building apis we're not just passing text around so what's really nice is you know it already sets up you know jackson for you so it can serialize case classes back and forth from the server so you can see that if you want to just rest we just return time from your class and it automatically serializes for you you go nice simple json um so managed objects uh these are basically just you know with most services your surface starts up you need to call start on a bunch of things wait for things to connect all right your server's running finally you do your thing and then when you shut down you need to safely shut down so so managed objects are just you know just that simple little classes that have start and stop and you register them with the life cycle of your environment in your application so you can just see you know so here in my you know sample application i'm reading from twitter just listening to a twitter stream and ultimately indexing it into elasticsearch so you can see this is all that i need to do to actually initialize hosbird for for reading from twitter you know have my client builder say connect say stop done um cool so now bundles these are just simple little initializers um and it's what's really nice is if you think about these basic applications if you just stuck with this simple application and just added all of your resources in one place and initialized everything together this thing would just become massive so what's really nice is you can just define these you know little sub bundles that uh just kind of encapsulate all of the you know the functionality that needs to to happen for you so you can have you know like you see here i have my tweet stream bundle where you know i initialize my you know my tweet stream my managed object i initialize a processor that actually processes the tweets i register both of these guys with the the life cycle of the server so as things start up they start as it shuts down they shut down and i'll talk about health checks later but i've also even just registered a simple little health check um so nice and simple and how this kind of works is you can just kind of layer up these these bundles to just basically mix in functionality um yeah and just so how things work you know it you can flip back and forth between a bundle and an application and they actually really have kind of just the same kind of signature you have initialize to do some setup you have run that you need to do right before you start the server um so they're they're just really just kind of like simple abstractions um cool and so to me one of the most interesting things about drop wizard is the admin interface that you get for absolutely free it's basically it just mixes together a lot of the the core java frameworks like metrics and setting up health checks and all these kinds of things to give you a very ugly but functional interface so as you can guess ping when you click it it says pong metrics gives you you know detailed metrics about anything and everything that you'd ever want when you click threads you get a thread dump of what's going on on your server and health check you'll get health checks and if you wanted to by default the admin interface you know fires up on a second port um one above the the the default port that way it's firewalled off your users don't have to worry about hitting it and it's just it's nice and safe here i'm running it on heroku so i only have one port so there is an easy way to kind of slap them together cool so let's let's talk about health checks um in any actual production environment you want to make sure that you can actually see what's going on with your service so it's it's really nice to just have it simple mechanism for for registering and dealing with health checks um so this is an example of what happens when you hit you know the admin slash health checks api you get this nice simple json response tells you whether you know are there any deadlocks i've registered two health checks one to check elasticsearch one to check twitter you know when something blows up you get nice detailed you know output from hey what happened uh so and this and this is all you have to do to basically just register a health check it's you just need to define define your health check have a simple method say check and return whether it's healthy or not you can actually return you know some some messages if if you want to um but for the most part whether it's true or false is good enough and again to actually just register it with it with the actual run time just environment.healthchecks.register elasticsearch and whatnot nice and simple so metrics again in production you really want to know what's going on with your system so you need to track everything there's no need to not basically not track anything so here you can see by default you get a lot of metrics just tracked for you so you get this nice simple rest api that you can write nagios health checks against or even just see what's going on when you're debugging your application um so you get all of these you know stats about the jvm you get a bunch of meters you can actually see how fast you're logging to your log files but one thing that's really cool is you get really detailed you can get really detailed metrics about your actual endpoints and all you actually have to do is slap on a couple extra annotations and you're good to go and you can start seeing um all right my hello world resource you know how many times has been hit how many times have i gotten exceptions what is the mean rate for the past minute and all those kinds of things and the best part about it is if you want to you know log it straight to graphite and build pretty dashboards and see what's going on you just slap this into your config file include one extra dependency to say how do i talk to graphite and you're done so there's not much else you really need to do cool so the admin interface also has this concept of tasks these are basically just one-off commands that you need to do from time to time maybe you want to i don't know uh programmatically trigger a shutdown of your service or in my use case i need to make sure that i have an elastic search index loaded and created and have proper mappings so you can just create simple little simple little classes again you register it with the admin and here i'm just all i'm doing is just making sure that i have an index twitter and you can just with curl programmatically make sure that you know any little thing that you need to do with your application is done so that's all really just kind of like the basic workings and from that you can build very simple applications is that a question so no not built so the question was is there any uh kind of like periodic tasks that you can do um so no not neces not built in but what you would do and something like that where you need like you know scheduled kinds of things um you'd create like a managed object using quartz or some other like background scheduler um to have these tasks and then you could you know create one of these tasks to just basically schedule your first one you can have it so it sits there repeats you can have another task to stop it and do those kinds of things so that's how i would probably go about doing something like that um so yeah so that's just all right five minutes cool uh so so what else does drop wizard give you um all of this was just kind of the basic stuff that you can you can get going it's you know it's not very sexy and functional but it's really simple um and it's you know it's using all of the the tried and true you know java infrastructure that people have been using for years and it's really google whenever you have issues that you need to debug but you also get jersey and apache http clients for free that are set up with thread pools and and automatically metric so that they show up in you know in your metrics api it has easy ways to deal with jdbi and hibernate if that's your thing migrations it's got its own migration framework for for migrating databases um i'm kind of bummed but it's it's based in xml uh xml is not my favorite but it does allow you to do quick little migrations from from you know just changing your database and safely and reliably authentication um you can also quickly drop in either oauth or basic auth and it works you can actually do views with mustache it's probably not the most you know inviting environment for for front-end developers but you can actually do do some stuff with that if you want to have a nice little interface and then custom commands which are a lot like tasks which basically allow you to you know from the command line run things cool some annoyances that i've used or ran into with it it's pretty java e and a little annotation heavy um it's a bit annoying to to manage like server specific configs across you know many different environments and but then you know that's really simple to deal with if you have you know an automated environment with puppet and those kinds of things and the one thing that i actually ran into recently it used to auto reconnect whenever database connections were dropped in the latest version it doesn't that's just something to keep in mind if you know just so you know um but yeah that's basically it so you just get a bunch of crap for free cool so to go a little bit beyond drop wizard these are two quick little uh libraries that we use to kind of make our life easier when building our you know little micro frameworks swagger it's just basically super simple api documentation you drop in again a couple you know a few more uh annotations you know i say this is an api you know and whatnot and then you get these really nice kind of web interface documentation that you can actually test out your apis see what's going on see what uh you know just basically see what's happening in production or wherever juke also a really nice you know sql framework um i've seen a lot of uh you know a lot of stuff of how to execute uh sql from from strings um but juke does you know type safe sql execution programmatically um basically it does involve a little bit of setup to get to a nice kind of clean interface but once you've you know once you've got your data source and you have a simple little executor class you can do very nice things like juke execute and build your your sql mappings cool questions anybody bueller yeah do you have to do anything special for the scholar support is it just really from a java perspective first uh so it is a little bit from a java perspective first the only thing that you would really want to you know make sure that you're doing with a scala version of this is to in your main application just include the scala bundle and that configures everything and make sure that jackson knows about case classes and any of the annoyances of trying to deal with java to scala but so as long as you've got that that scalable as like the first thing it's actually pretty pretty simple to work with yes so you would probably ultimately end up having to handle that yourself um which is a little bit of a bummer but you could just you know define your own custom metrics um to actually present what's going on no it would not intercept the features for you yes uh i don't know uh honestly i haven't heard much from coda hale about about the scala usage um it's it's been pretty pretty easy to work with um i've asked a couple questions on on the message boards and i've gotten responses pretty quickly um i can't remember whether or not it was from coda hale or not but but yeah it's it still seems like people are using it and it's it's rather it's just simple to work with so cool anything else all right thank you so much