Scale By The Bay 2018: Greg Kesler, Orchestrating Microservices with GraphQL
Recording: Scale By The Bay 2018: Greg Kesler, Orchestrating Microservices with GraphQL
you so about a kiss tracing the micro sources would have kill but we are not going to talk about the graphical how to set up the server etc this is not the talk about that before we start let me just introduce a little bit myself so I am a principal engineer the in treat at the entry developer group I've been working on the new generation of the intreat platform API and SDK for it I am a product owner and the TAC light on the SDK and on the platform aviation service that's what stands for a pass I also was contributing to the current API I was developing I was an author of the Siebel to a language that is used for creating services for the data which is basic oh and also I am the contributor to the graphical Java library open source project maintained by the convention so what we are going to talk about we are going to talk about what does it mean to orchestrate the requests and as an example of the requests that needs to be stated we'll be talking you will be using graph QL as the emerging popular protocol but it doesn't have to be limited only to graph Carol the problem with accusation will be common to any particle you can choose so we will talk about the what the what what does it mean to do the accusation it will also talk about relationships between objects we will talk about the role of the data is over sordid officials depending on what library you are using what are the problems where the current approaches that are popular in the graphical community stop working very well and also we will little bit about what the hotel solution we have Anton tweet to to to to avoid or to solve this problems just also to mention that we are one of the early adopters of the graph QL since it was open SPECT by Facebook in 2015 we've been already three years in the game so I think that our experience is also here so so let's consider let's say the problem so let's consider a period way that is connected to a set of micro services to talk to I will be using the financial domain about that there should not be a matter to to use any other domains as well so we have for instance a bunch of services like service that that provides company data that services provide contact data invoices accounting engine and the items Asian so this is these are just most important most common services or data types that belong to the financial domain and we have a bunch of a bunch of clients who use the APA gateway to access resources behind it the role of APA get way is to simplify access to the various micro services and various kind of data that company has so unify the API between them and also provide one of the important principle values the location transparency so the client doesn't have to know where the data is coming from the client just needs to send a request saying give me the data and boom the system will respond to that to the request and also it serves the purpose of optimizing the the traffic because all the necessary communications happen behind the Gateway there is well there will be minimum number of the quests over the slow Network and there will be as many as we need request behind it on the phone on the private fast Private Network so let's consider a graph you'll request that comes to the system the graph QL requests in again applicable to this the main looks like that the client wants to wants for the company company with Ag one two three four five fish invoices I actually the first one of the invoices that that belong to the company and it wants to fish those invoice that invoice or those invoices with a customer that associated with that so that's a relatively huge request if it was a let's say just rest playing the rest communication then it would be translated to a series of calls go and finish the invoice for this particular company then go in face a customer for that particular invoice and then the client would need to join them together what what happens here is after this bigotry achieves this request it decomposes the request and it organizes the calls to the downstream micro-services asking them for the particular data they can provide then it joins everything together and send that back to the client in the format in the form that shown here so the client receives back the object that contains the invoice and the customer not associated with it works so that's the role of accusation now let's take a look again at the request this coming to the system and the what the graphical engine does it basically builds absolutely cynics free from the requests the color coding here that the scalar fields are scalar fields are represented as the red boxes those are atomic data elements that live in one particular place green elements they represent objects and these objects are stored in multiple systems the this the son Helaman says those are arguments like ideas and ideas of the objects requested all the pagination parameters etcetera so and one of these fields or among these notes in this HT there are three important nodes which are the field node for the company field note for the invoices and field not for the contact why they are important because they represent objects served in different systems in different macro services so we call them relational objects and the field company means that go ahead and fetch some company data from the company service the field invoices means the same go ahead and fetch invoice from the invoice service and field contact means go ahead and fetch contacts from the from the contact service and then in and then also it implies that after the company is fetched then we know the night idea of the company we can go ahead and fetch invoices that belong to the company the same is after the invoice is fetched invoice contains idea of the owner of the contact and therefore we can go ahead and fetch the contacts from the contact service if you look at the same process from the perspective of what the standard engine is doing the standard agent was a standard reference Facebook or apology and a sec and the graphical java library that is basically a port of the reference of limitation to the java they associate with every field special object called resolver or data fetcher and when the one the one decree is being executed basically when the fields are being evaluated the corresponding data facials being cold or crisp of the solar is being cold that is all does the stays goes and queries the data from the from the from its service and then this data is provided to the Asian and Asian and it does all the joints well that's what the normal standard execution path of the graph query looks like now what is where it is falling short let's slightly modify the decree that was shown in the previous slide and let's add parameter to fetch invoices let's not only fish the first invoice that belongs the company but let's also specify that a criteria what kind of invoice we want to fetch and in this particular case what is the what is interesting is to specify criteria of to fetch an invoice by the attribute of the of the associated contact not by mu naught by the attribute of invoice itself contact and this is important I provided this example in the simple query language as I mentioned to you sequel like it's not standard Jeff keel does not doesn't spec does not say anything about RIT is so that means that you can want to define your schema in graph QL you can always add arguments that best represent your domain and in our case we chose chose to represent arguments to have the arguments that could specify criteria in that similar way there are other popular ideas by the way the idea of specifying expressions in the as arguments it is flying around in the community and who knows maybe it will be adopted at least we will be supporting that as well and there is another API which you might be familiar with might be familiar with it is called Prizmo api they have they defined for every craveable object the input type called filter input type that is useful in that sense that if you construct your query with the console like graphic ql that can do the introspection and the query it will have all the IG support or all the console console support on predictably with with predictions on what kind of what kind of features we can you can even specify but that's that that is not very much general here what is yell on here is this if we follow the standard ignition path of the graphical request just like we followed it before what and what we end up happening is that the without extra care the query for the invoices will go to the invoices over and that is over has only two options either to go ahead and try to a query invoice by customer attributes that invoice service doesn't have access to anyways or to try to orchestrate that Micra that create a query on its own and as far as it is there and we understand the to illustrate the critique way across multiple services outside of the graphical asian basically defeats the purpose of the graphical engine itself because graph collision is supposed to normalize the data is over they will ask only for the data that is required and then the asian will join everything together in this case it is either it is either the agent cannot argue status itself or the oil and therefore is older has to has to do some extra work and we feel that this is not necessary so why it is not necessary you can treat the each make a service in the system as the table and the the of that all the type that that measure service that service service company mega service service company objects and contact is contact - resource contact objects and the invoice motor service source the invoices and if you also also look the invoices have the reference to the company that's why that's how we anyways we are able to find the invoices that belong to that company as well as invoice has the ID of the contact that's how we can find the contacts that belong to a party to invoice and that's what helps us to to do the great research because in this case all you have to do is just click relate cartesian products between fetuses we need to fetch companies with the particular ID we need to fetch the conducts with a particular attribute and then find the cartesian product in order to calculate what invoice we need to fetch and presentation-wise and then motivation to join everything together and sell it all back so I will probably skip some of the slides as not that important I can only show another problem similar to that is mutation problem what if I need to create an invoice with the contact the same thing the graphical agent does not have does not decompose the input types it teach them as atomic types and therefore responsibility of our Cascadia mutations cascaded mutations lies on the four shoulders of the result so what would be the solution the solution is actually pretty simple if you look at this at these two problems from the perspective of who depends on what you might come to the idea that managing these calls with dependency graph would be helpful and that's what we do so for 40 query with the for the first example really clearly we basically build a dependency graph where each node in the graph represents a relationship and in this case we have the company service or company node that doesn't depend on anything we have the invoice invoices the dependent company and the contact and the contact that doesn't depend on anything else and that helps us to basically because this is the panetta graph to calculate the transitive dependency closure or to just walk these dependencies in the in the in their natural order and calculate which case we need to send in what order and what are the pairing we are able to select immediately to two creatives that we can by the way running parallel the query to company to face a company with a particular AG and face the customer with a particular criteria and then after we receive those we can go head and face the invoice because we have all the criteria to find the invoice and then after we resolve invoice we can actually produce result which is the joint happens in the result in the result note with the read number one similarly the class QD mutations are also could be solved we absolutely the same way with the dependency graph it's just the way how you build the dependency graph in this in this particular case in order to create a new invoice we need to have customer ID which means that invoice depends on the customer on the customer therefore we build dependency graph in such a way that the invoice no depends on the customer but after we build the dependency graph we we we traverse that graph or calculate the closure the same way as we did that for the clay and that produces us just imitation and we feel good about it we are about to deploy the combined service that that does not depend on the graphical engine to orchestrate it only uses the graph QL front-end to parse requests and some utility methods how to map input types on the own on the target language but the orchestration is completely bypassed it is used it is based on the dependency graph the good thing about it that we the same way we connect estate not only graphically requests but also rustic us Jeremy she requests any sort of requests that might that might come in to our attention in the future and that's basically it thank you very much for the reason [Applause] yeah sure precious [Music] we implemented the vandegraaff on our own we created the particular of ourselves and we just build a mechanism that builds dependency graph for for most for each of the partido keys for the quays and for the for the mutations and then we have the metadata that we derive from the objects by interpreting their schemas and based on the metadata we made decisions what kind of relationships objects have because we we can recognize that let's say parent child might be necessarily associated relationship where the player depends on the child or whatever so for instance in compose composition relationship where child depends on the parent I like wishes I think we love thank you very much [Applause]