SBTB FinaglCon 2015: Started from the Bottom Now We're Here

Recording: SBTB FinaglCon 2015: Started from the Bottom Now We're Here

to get started this is not a technical saber labs uses this is started from the bottom now we here if you're wondering why I've named this after a Drake track it's simply because we brought in finagle I think in a very different way than most people do we started at the very bottom at a very small team level and it has grown organically it was not a top-down decision so going on to that my name is Jeff Davis I work for Sabre that is a company in dallas if you don't know who's saber is that's okay well they would say it's not okay but sometimes depend on who I'm talking to they don't like that but they started actually a long time ago in the early 60s they came out of American Airlines dub we've been selling software into the travel industry for years and years and so the primary use case for them what they sell into our GDSS for mid-market and big market hotels airlines car car rental systems anything if you traveled from out of the city to hear whether you book the flight are you on a hotel chances are really good that your reservation lives somewhere in our system and so having said all that I don't actually work for the GDS group I work for the labs group and what saber labs is very much focused on is how technology impacts travel in the next two to five years so we do a lot of work with wearables we do a lot of work with 3d printing that's kind of more on our realm ex machina is a little beyond where we're spending a lot of our time on that's a little bit a little too far out for us right now and so in reality what that means though is we do a lot of prototyping the longest of the two years I've been there the longest that we've actually worked on a project is about two and a half months it's routine for us to have projects that lasts about one to two weeks it is also at least once or twice a quarter will have a project that last one to two days and so given that kind of very compressed time frame the initial thought is why you know why do you need to go to finagle it's a large tool built for massive amounts of requests etc and that's the totally fair question because for us in the prototyping world the reality is one request per second is not uncommon when you're building an MVP the reality is the business logic matters more than the high end throughput with this is a very unconventional use case for finagle in a lot of ways but start with these very small prototypes we want to get the business logic right and being able to leverage finagle has been has been allowed us to move it a much more quick and much more quickly so to give a little bit of a background here in the days of yore in 2012 when I first got started a lot of times third-party integrations were very small we wanted to do things like login with twitter or we wanted to be able to export your trip to a calendar application or something like that we wrote a lot of rails a lot a lot a lot of rails but of course even in the last two to three years the concept of an MVP is moved forward and so now we have to integrate a large number of third-party services even to do very basic Minimum Viable products and so one of the things that immediately became apparent to us real MVPs take performance into account when you're clicking on a submit or a calculator or purchase button a user might be willing to wait a second to a second a half a response but they're not going to wait 25 seconds for all of your HTTP requests to conceivably go through you know assuming that the timeout God's didn't chuck your request to know the lake at some point and so the most important thing I want to point out to all of this is that if you want to get started in finagle a finagle service does not have to be exported onto a network to be valuable for all of the additional hoop law that I've read about finagle and it's got a lot of amazing things about it now I don't want to downplay that at all finagle is fundamentally a tool to reason and a lot of times the reasoning involves a massive amount of throughput in a massive amount of requests but you were still reasoning very base at a very basic level and so if you are wanting to get started in finagle I would encourage you don't wait till you think you have a hundred thousand requests for second and a million requests per second to get started you can use finagle and you can use the finagle tools right out of the gate even within your own like very small applications and the reason that I want to point this out this is a quote by Edward Dykstra when he was at University of Texas in about 2,000 I think it is not only the violin that shapes the violinist we are all shaped by the tools we train ourselves to use and in this respect programming languages have a devious influence they shape our thinking habits and i'm going to add to that a small addendum so frameworks and libraries so if you're adding a lot of rails code every problem looks like a rails problem if you're writing a lot of finagle code though you're able to leverage that specific ecosystem to be able to reason that a lot more simple level and that's been very valuable for us we want to spend our time writing business logic we don't want to spend our time writing the same solving the same domain issues over and over again I'm not going to break any news here I think everyone here has copied and pasted code before when we first started working on this we did a lot of copy pasting we'd have a rails project that lived somewhere on github we needed to solve a similar problem we do what everybody does there you go to stack overflow they go to a previous code example they copy it out they paste it in they change the variable names if they're really into it they delete the code comments and then you know they compile and it goes that's an aggressive day but you know sometimes sometimes you're feeling good you wanted you would put that extra step in your in your framework and we realized that we were burning a lot of time doing this and a lot and just a lot of time we had very particular services we were rebuilding constantly and even if they were very low numbers of requests per second again we were still rebuilding those things we wanted to be able to leverage stuff we had written to stand up and work and be able to hit it much more quickly and so we started out by initially creating finagle services that within an application server and that seems kind of blah but we every service that we create whether it's an airport service or a hotel service or anything like that that we might use the domain that service always lives and labor business logic we have always lives in the specific app server for the project that we're working on and again these projects live anywhere from a day to usually two to three weeks but that happens I'll whip until it gets use twice so I'll give a canonical example of an airport we're travel where travel company we deal with airports constantly in some way form or fashion and so we got tired of writing the same airport service and so that eventually the phrasal uses we exported it onto the network we went from keeping that service inside the same application code base and instead that service now lives out you know in the ether on Aurora actually how we run it and we connect to that for Scrooge and so this is like a more complex version of what we might actually work and i'm going to point out at the bottom the middle part right here the appserver spitting json we use Finch for our application servers that's where we want to be spending as much as time as possible we want to be spending time writing the business logic to figure out and to solve your specific problem for whatever business unit that is and we've also got into the habit of we want to be able to spit very similar requests out to whether we're writing in react or whether we're writing on iOS or Android we want to be spinning out JSON so that our front-end developers we have a very clean break and it's easy to do easier to do testing it's very easy to mock a JSON payload everyone's done that and so up as you can see the next level will have an airport service and a ground service and location-based services and all the kind of things that you hit for services but one of the things that we do a lot is we hit third party aap is very very heavily so we've actually written finagle wrappers around every single one of these api's the twitter api we have a finagle thing that hits it parses the json will serve that back out same thing with a lot of the Sabre api's uber Evernote actually has a thrift API it's props to them they have heaven that was a little difficult to get working the first time but they do have that same thing with TripCase and Google's absolute smorgasbord of locations apps that kind of all seem like they do the same thing but you know they're valuable and so we hit all of those externally but we don't want to have to think about it and when I say that we don't have to think about it it we want to be able to respond again specifically to that middle component where is this business logic line we want to solve a problem and I'm gonna give a very concrete example this actually happened last week we had a request that came in to build a trip estimator app and we had people inside the company that you know you want to travel to let's say finagle con and you don't live in San Francisco so you have to go to your manager and you say how much is this trip I want to go and the guy says how much man a woman says how much is this going to cost so you have to go and you have to go to pile up and give a specific estimate that might include taxes and per diems and ground transportation some hotels airports and everything else and so we want to go saal baad but we actually we want to go be able to solve that and so we actually wrote that application the main guts of that application took us about five hours to because we were able to leverage every single one of the services required to be able to do that already existed that we had written previously we were able to leverage a previous micro art the micro services architecture to move but very fast but very correctly as well and so this I wanted in that particular instance I know this is going to be low to a lot of people here but if you're just getting started that particular endpoint to do the trip estimation would have been anywhere between 23 to about 28 HTTP calls depending on your authorizations you can imagine how long that could take in rails this with finagle and using Finch we did it in under about a second usually it comes in about 1.2 seconds and that makes running all the way across so specifically what finagle protocols do we use we use HTTP X if you were still on HTTP I would HIGHLY motivate you to go to HT TX I would consider it much more user-friendly to work with we also have a finial postgres we have one that we wrote that's internal that we're going to at some point try to merge into a lot of there's there's a separate finagle postgres that's on the finagle github group that is not the same as the one that we have worked with before when we get some time hopefully that may not be till the end of the year but we're going to try to do a little bit of emerging we also use i can't believe i misspelled it but we also use finagle Redis especially with clustering that's an additional protocol that we have on top of that and then we also use Scrooge to connect that's not finagle and I know that the core services library teams made that point very clear but you know it I almost into some ways I think of Scrooge and finagle maybe not as the same but the uses of them are very very tied together in my head what I think about reasoning in these things and so the next thing there's a nice Richard Nixon quote here but if you were wondering about how to get started in this if you don't have an initial use case this is a great Richard Nixon quote people don't vote for things they vote against them and so if you want to know how to get started with a finagle based system my advice to you is to find a project that doesn't work in what you're currently using that and that's exactly that's how we got on to finagle we had a high we had a system that we had we could make I think it was 88 parallel calls that can all be made at once well obviously we can't do that in rails so instead of just fussing about with that we said okay we're not even going to mess with this what's the potential solution and we went and that's how we made our way into finagle I cannot highly recommend finding something that people don't want to do as a motivational tool to move or to change tools because people get very in my experience people can get very kind of you get comfortable until I get comfortable with the tool you don't necessarily want to take that additional leap and then onboarding finagle I the group I work with I work with some amazing developers I work with two great rails developers and two front end devs neither no one on the team had worked on the JVM in years both prior to that so not only going to the JVM but going specifically to finagle itself was a big jump and so a couple of things I would point out for this avoiding overly magical syntax I I have admitted I enjoy the sirens call of the like Haskell side of Scala and all the combinatorial all the comp internal syntax and a lot of that stuff but for some of this just coming on a lot of that stuff can get very confusing very fast and you want to focus first and foremost you can always move as fast as your slowest developer when you're doing a lot of this stuff yeah it's just a parallel it's a parallel organization so avoid that's that they can always reason with what they're doing the second thing I want to also remember is very this has been my personal experience but in Scala every skull developer I've known that's come to it has either come to it for my functional side or they've come to it from an object-oriented side and so they understand kind of like half the language but maybe not the other half of the language and so it's always important to explain necessarily when something has been solved or you think a problem has been solved don't just assume that because the syntax has solved the problem that the reasoning behind it is not there that's just something to be very careful with but we have a I will say this in span of a little less than a few months we have actually earlier this week or last week I had a rails dev that wanted to write something in Finch and I said we probably should actually not write it in Finch sorry sorry Jason I'm sorry nervous I forgot his name but the but we actually do that they enjoyed that much so in only a few months we've people that are long time rail steps that have moved on and so that being said that's kind of it that is it any questions ok you mentioned also they was like rattling around couch ever you can what are the things that was that was actually we're very okay so on the back up the question was why what is the benefit of using finagle in this specific instance of as opposed to just having like a gym a ruby gem weighing around or some other code that you can bring on and the answer to that is interdependence see it we actually had that talk very uh we actually had that talk a lot when we first started working on this if your services are very very orthogonal then I can understand why then that doesn't necessarily come along for the ride but in our case we had a lot of interdependencies and so when you have as soon as that interdependencies ratio starts to move back and forth you have something where you want to stand up between the services excuse me you want to you don't want to have to like go chasing off which services you want we want to have a service that's up and we use it internally but we also those services that we stand up might be used by other teams as well and we don't want them to have to wait around and go through all of that issues we want to be able to post to a service discovery area and let them just use that so that's that's why availability and we don't want to have to futz with dependencies introduction yes yes I will actually I'm going to add a quick addendum to the other question a question earlier about why use finagle we've actually all was also found we are transferring over when we transfer over some of these MVPs to business units that may take them on it's been valuable to us to be able to say exactly what the expected when they're testing and building out a new endpoint if they want to take it over they have an exact understanding of what that service is going to return and so that's again that they can that's something that they can build off of a documentation in the case of finagle the import the service itself is kind of documentation and so they're able to leverage that to be able to move that to be able to move on that so thanks