Distributed Authentication using Reactive Programming in Scala

SBTB 2014, Bjorn Hamel: istributed Authentication using Reactive Programming in Scala

Recording: SBTB 2014, Bjorn Hamel: istributed Authentication using Reactive Programming in Scala

so my name is Bjorn and I'm gonna talk to you guys today about reactive programming for distributed authentication so just a little bit about myself in workday I'm a software engineer working on the platform security team at workday and the focus of my work is on authentication and Federation services things like Sam will open identity connect oh aw - and workday is a cloud-based HR finance and stop software as a service provider you know I think our marketing tagline right now is HR and Finance apps built for the future and what this means to me as a technologist and on the tools and platform side of things is that we're building these new platform services using modern technologies and tools and in a lot of our recent efforts we've moved towards using Scala and akka to implement platform services to build out these HR and finance systems so why go reactive for authentication why not just stay the course of our traditional mechanisms what I've been focused on in particular is breaking out our authentication logic from our sort of traditional monolithic embedded approach to a completely standalone service that the platform would be able to leverage across many points and in doing so we've moved from our sort of traditional stack of using custom Java on Tomcat towards using Scala Anaka and in some cases running with Play Framework and in doing the system redesign we found a really compelling case for using reactive programming I think a lot of our initial interest in Scala was you know was an elegant way to solve our problems it was more of a joy to write code in Scala but for this particular use case no one on my particular team could really refute why we wouldn't choose a reactive approach over our existing Java based approach so I really wanted to share those results with you guys show and compare and contrast our previous implementation to this new implementation so how are we gonna do that we're gonna first look at our use case called delegated authentication we're gonna briefly explore the initial solution we had using a Java based thread pool and the interesting part about the the solution is not when it works but when things go bad and so we're gonna see how we can fix that using an asynchronous non blocking model I'm gonna do a quick demo sending lots of requests on it and see what happens and then finally sort of explore why this problem so much worse in multi-tenant multi-tenancy models but first I'd like to show you the vasa what you're looking at here is the can index of the vasa this is a Swedish ship from the 17th century and at the time had the largest broadside firepower of any ship of its class at two levels of cannons and in really at the time you know the tactics of sea battle were changing and this sort of drove this this cannon innovation but unfortunately shipbuilding in architecture was an imprecise science at the time people had a rough sense for what to do but they didn't have a precise model so rather unfortunately for this ship thirteen hundred meters into its maiden voyage the Vasa sank so what happened here the sails cut a strong strong gust of wind and these water level cannon dicks deck this innovation they were open as the ship was leaving the harbour and they began to take on water and that was sort of sort of the end of it for that so this design that was still focused on military might sort of eluded the basic functionality which is you know your ship needs to stay above water and really what led to that is a spiking conditions right it was the fact that the ship was in a particular configuration and experienced just too much wind at this at this given moment led to the ship completely failing so by way of thinking about catastrophic failure in systems I'd like to introduce our protocol that we have to implement called delegated authentication so in this model we have a user using one of various browsers they'd like to login or authenticate to our system so we have a presentation tier that serves up some sort of login form to gather a username and password and then that system takes those credentials and relays them on to a back-end for processing and what's interesting about delegated authentication is we don't actually store the password hashes for these users which is actually desirable from a security standpoint you know we don't really want to be responsible for those in any case so the customer is able to configure their system to relay these credentials out to an external endpoint so you have to send the credentials out across the interwebs to some sort of RPC endpoint some web service that's exposed where they've implemented an API saying sure we'll take the credentials and we'll check them for you so maybe they have a few boxes configured to do that and those boxes usually actually can't answer that question either so they have to do a query to a database or maybe an LDAP bind to actually verify that those credentials are correct so now you see one two three four or five points of failure potentially two answers to this sort of relatively simple question can i authenticate this user and unfortunately three of these things are really completely out of our control we have to rely upon these systems to answer this question so how can this fail well the network routes could be slow we could have some issue actually getting to that external endpoint the services that are hosting their web service could become inundated with requests this could have been sort of post patching and they're having some issue coming back up to speed or the database that they're hosting is inundated with requests it's unlikely that they're going to turn up a dedicated database just to perform this service so maybe they carve off some small portion of an existing system and it's just not able to keep up with the load of sending requests at it so the initial solution that was done in Java was basically to use a thread pool to sort of encapsulate the amount of effort we wanted to spend attempting to make delegated authentication requests so over here we have our thread pool and we use a timed thread pool so we sort of say well we're willing to wait number of seconds to make these requests and we're willing to make n number of requests in parallel and after that we're gonna cut off the request and say it failed and I know this is not a Java conference but I just want to give you a sigh a sense for the scope and scale of what's required even just to make a toy implementation of this so basically to expose the thread pool to instantiate a thread pool of ten threads to taking some parameters from the request create a new caller Bowl say how long you're willing to wait for it handle the various exceptions and exceptional cases you know it's a whole screen full of code and we actually haven't even done any business logic at this point and this is all just bookkeeping to set up our thread pool and finally we get to our callable implementation where we can actually say okay we're actually going to make an HTTP request we're gonna make a get request to this endpoint that will actually you know represent the customers ability to verify the credentials and well execute our get request and at this point right here when we actually execute that get request we're gonna block on that external site so I'm a very visual learner and I wanted to sort of you know imagine what does this look like in practice when things sort of break down you know what happens when things begin to block so okay and in our hypothetical scenario one of those nodes becomes overloaded and now it's taking five plus seconds to respond to our request this is a particularly nasty kind of failure if we had to immediately fail that would be great because we could have immediately turn around and tell our end user sorry we can't authenticate you right now but we think maybe they could actually have the NICUs we just have to wait this five seconds for them to reply so as a request comes in from this presentation tier the tomcat thread pool allocates a thread to serve it and it spins up this thread in the in the dello thread pool and it starts to block waiting on that external site and as more requests come in they start to stack up on top of each other you know we've got this timer to try to call this this behavior and limit the impact of this but if we get too many requests flowing in too rapidly we sort of starts to see this this one-to-one parody of Tomcat threads in the container and sort of the ability to service new requests coming into the application being all tied up behind this thread pool waiting to execute and so you know we need an alternative approach to this well we can't this isn't a serviceable model having the blocking leaking up to the tomcat threads in particular once everything's blocked waiting for you know delyth executes then other requests for users who are already authenticated aren't going to be able to be serviced either those look back up blocking behind that as well so instead you know and in looking at refactoring to create a central service to handle authentication across many many tenants and many many systems we especially can't exhibit the behavior where one bad actor in the system could could backup the entire model you know we need to be able to scale with the load that we experience so this is where we started to look to reactive programming and I attended a talk earlier this year James Ward gave a great talk at the SF Scala group called reactive all the way down and I was I was taking Bart back to the East Bay thinking about you know the ideas he had put forth they sort of clicked that wow this you know this is the perfect solution to the problem we have at hand you know we have to rely on these external parties to serve our requests but how could we do so without consuming as many resources so enter reactive Dell off so in this model we have the same scenario we have the user authenticating to the front door of our system we have the external service that we're relying upon the exact same external service but now inside the guts of our giant computer we have many threads ready to service the requests and it's dispatching those requests asynchronously is in the plays default thread pool and it's creating futures to block on each one of those get requests as it sends the external requests to the service and those those futures are running inside the execution context so we're now you're actually creating threads we're not having to manage a pool of threads and instead we're allowing the framework to manage the futures that we create to execute that and so it's using its own threads to do that control so what does this look like in code in play we need to create a route to represent that so for this example we're just going to define a get request that says we're gonna invoke this dello service and the implementation for the service we're gonna have to create a controller so we define a controller and then we say that we would we'd like our action to be asynchronous and then the actual definition of our of our action is we're gonna extract a username and password parameter that we're sent in on the request and when we make that request we're going to use the Play WS library which actually makes an asynchronous request it doesn't immediate it returns back a future for us to wait on the return from and so in this case it returns the future of type WS response and then we're using another trick here where to sort of get better sense of control for timeout we're creating a promise that says we're gonna execute this request and we're also going to wait in this promise for 60 seconds and then we're gonna compose those in a sequence and wait for whichever one's going to complete first so if our external request completes first then we'll write back the response of what the service returned and if we haven't gotten a response within that 60 seconds then we'll return up you know a 500 error that we timed out so what does that look like you know in contrast to the model that we saw before so the same system that we're trying to talk to is become inundated with with traffic it's overloaded as taken five seconds to respond so a request comes in we have a socket that's consumed when we make the ricotta in Bound requests to the server and for a moment the nettie thread is blog it has to read HTTP requests in and sort of relay those bytes and figure out how to route the action and as you saw in the code in the previous slide this Futures created to make the get request to talk to the external service but what's markedly different is as soon as that future is scheduled as soon as it sits it's begun executing and we're waiting on it Nettie's it able to immediately use that thread to start to handle other requests it's no longer blocking waiting for any more i/o its fetched all at i/o as far as its concerned it's it's done waiting so we have this open socket to our server and we have the future waiting on executing and getting a response back from the external system but we don't have any other resources consumed and as we get additional requests coming into the system in parallel with that first one more sockets are created momentarily that the nettie thread is consumed reading the request before it it creates another future once it figures out and you know the action to execute and then again the nettie threads give them back and so what's particularly interesting about this model is we don't end up with fixed amount of resources that sort of have this upper bound that are going to be consumed against we don't have this thread pool that only has ten threads in it or however many threads that we have to figure out you know what's the appropriate number to pre-allocate we have dimensions that we can grow in we can create additional sockets to handle additional requests as we need you know up to the upper bound of the number of file handles we can create on this particular server and we can create far more futures in this execution context that we can't then we could threads before you to use a threading model besides the fact that we don't have to know beforehand how many threads we might need there are much lighter weight entity and so we can scale in both of these directions with the amount of load that we're experiencing also because we're not blocked behind any particular resource in this case let's say that our customer actually had a load balancer and they had multiple nodes able to service this request since we're still able to send non blocked requests through we have this open route to host that's available to service our request and maybe that's able to service our request immediately and so by using this model we're able to achieve much greater performance in these failure cases so what I was curious to see is now that we have sort of you know two potential models to compare you know our previous approach is in thread pool and the Navy base play approach is sort of you know race them against each other and see actually how they do perform in action so we sent you know n requests into a simulated delegation authentic delegated authentication service and we had half of the requests go to this one node that's experiencing this partial failure so I was waiting for five seconds before it serviced the request and the other half go to a node that was running you know quote-unquote normally and returning after 100 milliseconds so in our thread pool based model we use ten threads each timing out after fifteen seconds and in the async non-blocking case we had a timeout set to sixty seconds like you saw before but before we could actually run the scenario we needed some sort of tool to send lots of load at our and our simulated scenario so we did some looking and found a great solution using Gatling Gatling is a wonderful open-source tool that allows you to use Scala to define load tests for your application it uses asynchronous HTTP client and netti libraries as well to send out those requests and it provides a really nice DSL to write load test your application a nice fluent DSL so basically what you're able to do is define a scenario you can define HTTP request so in this case I'm making a get request to a congested endpoint I'm going to make sure that it returned back with a 200 status code I'm going to check in the body of the request to make sure the string done showed up and then I'm gonna make a subsequent request to the normally performing endpoint again making sure it returned status code 200 and then it returned done I could pause for some random variation and then ultimately I say I'm gonna send 50 requests in in this model so that'll be a hundred total requests one slow server and one to the normal and I'm gonna ramp up rather quickly and just send them in one after another only wait in a millisecond to add each additional user to the system as a bonus it renders really colorful charts so if you have to show other people how things are performing at least you get that so that's good and the default climate timeout for the system is about a minute so that's another detail in case you want to change that configuration so we're gonna first demo the blocking approach all this is running on my laptop say basically we're gonna run the gatling client I gave against the tomcat based implementation using the java thread pool and then sort of from that middle processing to here we're gonna send half the requests to simulate an external service that's performing normally and half of the requests to a congested node so let's hop over ok so we can see hopefully you guys can see that so we're gonna run Gatling and it sort of goes and looks and sees all the scripts that you currently have defined so we're gonna hit the thread pool based approach and asks for a couple default parameters we're gonna show to ignore them and starts up the scenario hopefully shouldn't take too long and so you can see it gives you some great ASCII stats as to what's going on we're getting a little bit of line wrapping because my font so giant and then we start to see a bunch of failures fly through here and it's done it since 100 requested the service and it gives us a nice little status report that we can go grab and throw this in our browser so big but cool we can get a quick indication 70 requests made it through 30 requests failed and we get some quick statistics on how they executed we see that the requests to the slow end point took on average about 1112 seconds and the requests the endpoint that should have responded in 100 milliseconds actually took on average about 8 seconds to respond for those that successfully completed so if we hop back over here so to make a couple quick T tweaks to the Scala based implementation before we can do the same thing so we had another route and play basically breaking out our existing service to call two different services one to the congested service and one to the normal and sort of we wrap our service invocation into asynchronous actions and send the implicit request context along down to that so that you can pull out our parameters and again same code as we looked out before it changed the two features together from a timing perspective so just to be really clear this is the exact same model same client except written to call the specific endpoints here we switched out the middle implementation for the code you just saw and we're calling the same two external services so let's see how that one executes so I go to run Gatling we have a scenario right and hit the reactive model oh yeah it's alright thank you silly computer science certain zero okay um all right so we started the reactive model and we see it finishes nearly immediately we can open up the chart here and we can see things look a lot better we have no failed requests in this case and actually a large number of our requests replied in under a second so we see one hundred percent of the requests were served and on the average hitting the slow endpoint took the five seconds as we expected and really much nicer much better in this case is that it only took one hundred milliseconds to serve the requests to the normal endpoint so we were able to have direct communication to the server that wasn't bogged down because we had nothing blocking in the way so that was fine and good for a hundred requests but we're sort of curious to see what would happen when we ramped up load against this model and service more requests so in this case and you know initially we sent 2,000 requests in a couple of seconds and sort of saw similar failure behavior in the Java based implementation over half the requests fail and the average times are horrible thirty seconds on the average the slow service and particularly disturbing is to the service that was sitting there ready to respond in 100 milliseconds we got an average performance time of 30 seconds talking to that endpoint so in stark contrast to that with 2,000 requests at the reactive node we see again a hundred percent of the requests are serviced in this case you know we got sort of this throughput of like like two times the previous one and what's really impressive you know again is the average wait time for the slow node takes up a little bit we're up to 19 seconds now but for the endpoint that's ready to respond we get this immediate response of 100 milliseconds and that's you know far greater than the other scenario oh so we wanted to you know continue to play this out and send more and more traffic at the different endpoints and look a little bit you know in our scenario as to how they perform when compared against each other so what we see here is for the reactive model so what we're first looking at is response time by the approach for the particular node so across the board no matter how much traffic we sent at the normal endpoint we always got around 100 millisecond response time which is fantastic for going against the block node using the reactive approach this beige color here we saw you know a similar similar pattern around 19 or 20 seconds and then using the thread pool model in contrast you know again things capped at about 30 seconds across here whether we talking to the normal node or the block node and what's particularly interesting is the failure rate percent by the approach so you can sort of see it out of the gates trying to talk to the block node its timing out obviously and you know we get this horrific over ninety percent failure rate even with only 2000 requests and as we ramped up to 3000 and 4000 requests you know the wheels completely fall off for the thread pool and it's it's pretty much useless at 5,000 requests over a few seconds and what's interesting here is is for the reactive model you know things continue to look really good up until they get to around 5,000 requests in that given few second interval and then we actually do see a small failure rate about 12 percent of the requests did fail and going against the the slow node we saw that you know even higher number of requests started to fail around 40% when we sent 5,000 requests in this interval so you might think to yourself you know what the reactive approach can't fail you know Scala can't ever let us down but in this case what we see is that we hit up against the file handle limit of the operating system so we see a bunch of exceptions being thrown too many open files in the system so by default you know your Mac has about 10,000 Kay limit for open files and so as our client and our server running on the same system made 5,000 open requests and tried to service 5,000 requests at the same time and then we had a few other you know background things running we sort of tipped the system over and hit that 5,000 or hit that 10,000 number and for the slow case when we started to see failures in the 3,000 to 4,000 model what we saw is that for the slow nodes we actually hit that 60 second timeout and gatling so you know I'd mentioned before that the default timeout is a minute and while it was executing all those requests in parallel and switching from request to request to see if they've been serviced or not it actually did take longer than 60 seconds to service them so this is just a side note on Tomcat configuration and if you can see this here but basically before we change the setting the reactive approach and the tomcat approach behaved identically whether we use a thread pool or we use reactive programming and this is sort of an interesting point to consider is you know we're talking about mitigating failures when we're communicating with external partners potentially people out running servers outside of our control and so what we had to do is we had to greatly increase the accept count on that Tomcat service we had to bump it up to like you know I think I had 10,000 in here if you run a vanilla tomcat out of the box it can't accept that many connections in parallel and basically it starts to reject all the inbound requests and you get you know again identical failure rates so you know while this is a much better solution in the generalized sense and likely you'll be consuming services configured to sort of you know work at internet scale if you if you are talking to a smaller or federated with a smaller entity you might have to make sure their servers are properly configured to handle that many requests in parallel so finally I just want to talk a little bit about how this failure potential compounds and multi-tenancy you know as we saw in the previous examples you know we had one processing back-end and that went out to one customer site but in reality you know our systems are multi tenant 'add and we have one processing back-end that mat might host you know tens or fifties of of tenants all in parallel and need to share those same system resources and so if that happens you know whether we use a thread pool per tenant or some other model to mitigate that if we do have this case in which the blocking can leak up to the tomcat threads then we really need to use an alternative approach to sort of defend against this failure rate so you know again this one service going down and experiencing a high load to authenticate customers to tenant a could have this ripple effect if we don't have some way to compartmentalize that failure so in closing I just wanted to highlight that you know we didn't design to say how can we make the service go from you know four seconds to three seconds or how can we use you know less memory or how can we do X what we focused on is how can we compartmentalize failure and when we did that when we were able to achieve that we sort of experienced an increase in performance as a side benefit the the Scala play based approach allowed us to focus on business logic and security details as you saw from the two code examples we had an entire page of code just to establish a thread pool and some some bookkeeping and all the boilerplate surrounding that and in the play approach we were able to focus directly on the implementation of extracting parameters from one side and sending on our request to the other side what we found with the real key to performance in this model was that we left open pathways to the responsive nodes so instead of instead of blocking on this thread pool and you know not knowing whether or not we could get a response back from you know the slow external site or not having the ability to continue to service new requests as they came in is really what yielded the greatest performance and that's where we saw the ability to continually get this hundred millisecond response time from our node that was ready to service requests because we didn't have to stack up or wait behind these nodes that were slower so you know final thought here is that you know great tools and techniques allow you to test your ship before it leaves the port and the right frameworks and architectures ensure that it doesn't sink on the way out thanks [Applause]