SBTB FinagleCon 2015: Alexey Kachayev, Finagle and Clojure

SBTB FinagleCon 2015: Alexey Kachayev, Finagle and Clojure

Recording: SBTB FinagleCon 2015: Alexey Kachayev, Finagle and Clojure

it's room for sure is that pray that I'm in run audience but today i will show you few slides with closure code and if you don't known closure please don't worry it's super simple it just does what it says directly and I've shared a link to my slides in Twitter visiting angel cohn hashtag so feel free to follow me from your laptop usually it's much easy to read code from your own screens few words about myself my name is Alex a tkachev I'm city or identified identifiers san francisco-based startup will build a platform that empowers thousands of event organizers to create private social networking applications and this application usually help to boost engagement during your event catalyze learning and help sponsors to connect with attendees and this is a screenshot from our private social network actually one of thousands of private social networks that we run and as you can see it's just the full-featured social network application with profiles activity streams messages photos likes comments and more and more and more and most probably you saw such features before if you use such tool as Facebook or Twitter and I believe you do what you definitely can see on this picture is how complex and reliant data layer is and to deal with this complexity we use microservices really heavily we run tens of them we use scallion engl are from few core services but most of our cut the base is written enclosure and we also use closure screen for most of our front-end development to run closure in browser so chenango actually covers most of our problems with micro service infrastructure that's written in Scala but enclosure we don't have such framework steel which is a shame but we still don't have such framework and not even close there and there are few separated libraries that covers few different parts of you different problems to some extent different ways you can check many of closure conference talks about microservices architecture and you will see that there are only few little steps we're done in in this direction and still most of scenes left just uncovered so here's a simple question can we use been able to empower our closure microservers infrastructure and the short answer is yes and here is just amazing project created brand by engineers from twitter which is called finagle closure and it was presented i think last year on strange loop actually this is just a thin layer around finagle that solve most problematic question with scala interop from closure on time you see a clue most probably you know that Scala has amazing support for giant robe and closure has amazing support for Java interop as well but inter up between closure and Scala it's like very tricky stuff and actually if an angle closure covers all problems with this question and to get more insights from the author or this framework you can check video from strange loop talk and you can read quick start guide from github i will i will give you just a rough idea how it looks like and we will go or dive even deeper so if an angel closure is shipped with a nice line template a lame and mature how about the prices correct Lane enclosure is something like SBT in Scala it's our most common and widely used build tool so this template is super neat because you can just create up and running server and client we spin angle just in few seconds and then explore different scenes just experimenting with already generated code for you if you will run this line you and checker and directory you will see that it will generate for you three separated closure projects one for client one for servers and core which supposed to contain all libraries that will and all code that will share between two previous like for example your three definitions so let's define simple three a thrift server this is just standard rift file as you probably saw many of them before with the single service with the single single or defined function in the service and you can compile it using using screws compiler which is also nice nicely integrated with lame and hear how you can create servers from closure you need two packages futures and thrift both adjusts in wrappers around appropriate Scala functionality and here you see that we are creating three servers from compile schedule definition and this is fetch session this is function that takes a single argument with the request and it returns a future just well you lifted to the future so thrift serve run server for you and given port or you can use announced functionality from 32 to work with different service discovery mechanism this is client implementation this is how it looks like it also really straightforward everything that you need you have to call this thrift client give address where to find server and give compiled by Scrooge scheduled class I hear how you can use this in your code this is really simple example we have fetching session with x given ID and then we met him over it to take a title so this is future map transformation and here's a slightly complicated more complex example to find the speaker named by given session ad you have to fetch session for the first and then find speaker ID in the response and then send request to the client with fetch speaker function and then map the result to take first name from the response so you will see here the both both transformation with flat map and map and i'm sure you all know how they work and this is how they look like from closure so again good news for for decent moment it works it actually works and you can catch me after the talk and i will show you this up and running on my laptop just in to see this in action and it's amazing and I really appreciate the effort that was put to make this happen but there are there are few drawbacks few problematic seems that we have to handle somehow and the main one is that Scala API is not idiomatic foreclosure and it's really hard to deal with justice in rubber and you know this is really fun that you can take almost any finagle based go would any anything that you can find for example in Stack Overflow or somewhere on the internet and you can just translate it almost word by word into closure and runnin but it's really hard to keep sane when you are dealing with other closure libraries or when when you are trying to introduce this functionality in in a big project that already exists and so are there any way to solve this problem and we seem that yes and i will show you few steps that we think is just right there just little movements into the right direction and let's see what problems can resolve first of all let's start from scholar futures and scallion twitter futures they do great job in scala ecosystem because most part of this ecosystem is built around them but definitely not a closure and the problem is that two branches of execution with success and error and not idiomatic foreclosure and error scanlan based on exception is almost not unuseful but it's really painful to deal with and also managing executioncontext is really painful and I'll show you better ways how to deal with chaining of async async callbacks that more useful enclosure there's then example that I showed you previous slides and just don't get me wrong here it's not about the right or wrong way of doing things things is just about playing the same game with the same rule so here I want to point out that the closure has many native concurrency primitives and languages say itself was designed with concurrency in mind but we're not going to move on with built-in primitives like futures promises edge and spin apparel reducers and so on and the biggest problem is that they tightly coupled with the execution context and it's really hard to manage both run times and the same time so what libraries can we use there are two of them first libraries manifold and this library provides basic building blocks for a scene programming and it can be used as a translation layer between libraries which use similar but not but incompatible abstractions which definitely sounds like what we need and the main manifold abstraction is stream but it also provides you with Deford well use like some kind of special case of streaming and definitely looks pretty the same as scala promises you can wait until the values you that you can zip our feud efforts into a single one you can chain callbacks and so on and hear how it looks like this is how we can convert a scholar future into closure definite values just the same API on success on future just a yield value into debt and failure just propagate exception it's super simple you can see here how we can wait until until well your is ready to be used and here's example of chaining and so we can just rub this into a simple function and then rubble of all of our calls to client with this just future to defer transformation and what did we gain here or why just why should you even bother about additional layer of indirection and this is the answer so we all know that callbacks are useful building blocks but it's really painful to build sophisticated workflows I mean a sink work flows from separated callbacks and we all know what called back he'll even and here is sled flow marker you can see it here and this macro gives you ability to write simple and concise code the chain concurrent operation that will rank in running a sync mode so let me do this so the the same as we did for let you you see there's three different bindings but in inside lead flow you can treat all of your death rate they were already realized so they're ready and this led flow macra it looks something like for yield for Scala but it much it's even much smarter because it will build the whole topology of all of your operation it will find independent parts it will run independent computation concurrently and then collect them into like stage by stage execution and it will return me another definitely you can wait before it's done you can obviously you can do this with the zip and chain operations on map and flat map but in this in this case it will be straight forward but however the code that you can see in this slide is just a very direct expression of what we are doing and what do we want to do in the end grazing is another really popular library and it adds to closure facilities similar to goal and programming leg or teens and channels and if if some of you know what you sharp is this like a sink f wait to some extent and actually it brings CSP programming style as a library and it gives you this nice goal macro that will compile your code into some some sort of state machine i will show you further so here the same the same stuff how we can convert a scholar future into channel on success and failure the first thing that you should mention here that we create and channel with there with the one buffer of the size one so any put operation to this channel will complete immediately and another scene that you have to mention that we use here I the left and either add right abstractions because we don't have this another branch of errors or exception propagation so we have to explicitly tell if this successful computation or this same kind of failure so here you can see the same example with speaker name and using goal macro and hear what it did it's a bit more stuff that we should write here and it's more explicit we should define so we want to read from channels but the amazing stuff about it that it will recompile it to some sort of state machine that will that will wait until our this operation is done before going to to next stage so this is how you can convert your channel back to future if you want for example to to give answer for your server and we just use here scholar promises and just here just set value the first values that we can take from the channel just resolve promise with it so in general it gives you clean and concise is in code but the key point here is that it's an open door to their closer async world because the most libraries enclosure either use a croatian control we'll provide you some kind of bridge just to be compatible with it so the next part is more codex you know the thrift is really nice but sometimes it's in the work overkill sometimes it doesn't solve all your problems and for example in Scala we have finial sorel to work with escada library which is amazing enclosure we have ration and freshen this is like a binary notation done in in spirit of chloro and it's binary representation for this format that's called Eden extensible data annotation and everything enclosure is done in Eden including closure source code itself it's it has many many types that you can see here like keywords numbers strings vectors sets maps and so on but their most amazing part of this notation is that it's it provides you these tags names with pounds and following with any Eden firm that this one for example this built-in and this directly tells you that this is this is not extreme this is daytime object so now we are currently we are working on freshen codec 15 angle and most probably we will open source it eventually okay I as fir SE C we ran out of time and just fewer Xin conclusion finagle and is great and closure is amazing and and finagle can solve a lot of problems for closure ecosystem but integration still requires some lot of work to be done and I believe that eventually we will do all of this work and I'll be there till the end of the conference and so feel free to pick me up and I'll be glad to answer all of your questions and thank you you