Scale By The Bay 2021 : Viktor Gamov, Proxies, Gateways, and Meshes: Cloud Connectivity 101 for...
it's pretty it's pretty awesome confidence it's always great to get back at speak at the scale by the way uh good morning everyone um thanks so much for this for for this introduction uh and today we're going to be talking about some of the patterns uh different techniques and ideas and hopefully at the end of the talk you will be quite savvy on understanding what is it what's the difference and um many questions that i have usually they coming from this like what's what's the difference like what's the uh what's what's this all about and uh today we're gonna talk about this so i would like to start to talking a little bit about um like why connectivity is important and like why having uh clear and well-defined connectivity for the services it is important and uh it's part of the every every engineer agenda i would say so everything is easy when you have just like one service or one component or one application this application uh does one particular thing um you're starting maybe with some sort of mvp like some of the idea that you have you start building this idea and build this as a you know beautiful piece of code that works and does exactly what needs um but during the time when you need to start moving these things to uh to production you start need to think about some of the data responsibilities and in many cases those things can come with a price you start building complexity in your application you need to provide things around authorization notification uh maybe using like third-party service to do this uh maybe providing the ways how your service will interact with other services and so far and so on your your application is not that small anymore it's uh it's already big um there's a lot of functionalities there's a lot of um maybe somehow even indirect communication between different components and between different services at some point you need to bring some of the external systems into play things getting really hectic you don't um already don't see light you don't really understand what is going on there and uh the management of these things because you keep adding the code and this is how we end up in a situation where that many people know they're known as a kind of like monolithical application where it's not easy to think about this and it's not easy to um what's the word i'm looking for so it's it's not easy to reason about this uh microservice or it's not the microservices monolithic service um a bunch of that code um you need to have a crazy mad scientist who will explain this to you and specifically application uh you know how the services communicate to each other things becoming not very easy not one person can explain this and i think this reference um to it's always done in philadelphia it's always great explains like how when you're trying to explain how the services communicate and the different apis calling another api and so far and so on so we'll try to make sense out of it by implementing some of the clear patterns of communicating of the system and at the end of this presentation i'll give you even a quick cheat sheet that explains or how your mental model should work i work as a developer at the kit at the clunk at conc we are building solutions for cloud connectivity um i am speaking at the conferences since i'm developer advocate i'm trying to meet developers where they are so we can talk about the things what they need to know if things what uh they need to build uh you can find me in all these places twitter is probably the easiest place where you want to ask the questions uh also i'm available on telegram and also i have a youtube channel where i posting some of the videos and streams and talking about stuff um and the quick announcement for the people who seen some of my talks in the past yeah i was doing a lot of work around the kafka and this work will culminate in an upcoming book called kafkin action um i just got the call from manning um like yesterday that uh they told me that the book will go into print the literal next week so i'm super excited and you can grab the code kafka a kafka in action 35 to get the um to get the book right now so kafkainaction.org check this out so um we're going to be talking about the first place we're going to start from explaining the concept of probably many of developers already familiar with concept of proxy it's one of the patterns that people can use in their application but also it's a infrastructural pattern that allows you to put something in front of your application to [Music] to do something in many cases uh back in the day we put the proxy in front of our application the plurals you might be doing something like caching of certain resources and this proxy will be responsible for um for routing traffic to your application and the more and more we start hearing the uh term a gateway more often and what is it and like how it's different from proxy so in this like first part where i'm going to talk about you we're going to focus on api gateway concept um and um one of the things that i'm mentioning here now the the sideline during the simulation new capabilities will provide at no cost um also like you write down in the comments or like if you're watching this live uh or like in discord like if you get the reference but the reference would be explained in a few slides so just a quick quick note all right so what's the api gateway anyways right so we know that's pretty um a pretty familiar concept where the um people might say okay so there's that's maybe just clarified proxy or whatnot so api gateway pattern describes a some somehow additional hub in the network where your clients communicate through some service um and uh in this case it behaves as a kind of intermediate thing uh and uh that's pretty much it you can say that's that's that's what we have here that's what we um what we used to know as a broken getter what's what's the difference and uh um i feel like this explanation it's kind of a little bit of uh simplification of this uh of this concept uh sometimes it can be called proxy sometimes it's called load balancer but um i think the api gateway it's not maybe a particular product or project i think it's just like a pattern and uh i would like to break down the i think there would be like three examples where we can use this um api gateway and the first pattern that i want to start with is the service connectivity because this pattern essentially like is how everyone is using api gateways it's a kind of good all day good all day proxy so in this use case it's about providing or like enforcing some of the policies that your application um needs to employ on the previous slide so in this slide you can see here um this this part is super super simple so our our client calls this is our say client application so we need to call this uh this api but we're not calling this directly we're calling this through this proxy so through this guy and since now traffic control belongs to this intermediate intermediary this like api gateway we can do a bunch of cool things with this specifically let me get back here so specifically uh there's a lot of interesting things can be done many of people um offload some of the work around the traffic management to uh to to api gateway specifically uh providing the ways how client can communicate to underlying storage through https for example um in this case developers of um this like api that we need to call they don't need to even worry about this and implementing this as a part of the software out of the system in this case the api gateway will handle this and providing this um out of the box so this piece of functionality will be available next thing that um you can also implement some more advanced uh um like traffic policies um like things like um a rate limiting um your api might be calling some underlying systems that you really don't know and we talk about this also as well and those calls can be very expensive and when you're calling those systems you don't want to overwhelm this by incoming requests and another example which will be correlated to next pattern of using the api gateway the api as a product you might want to restrict or provide a different level of service to your consumers to your application consumers there might be some uh customers that get access to your api for free so in this case you need to provide certain level of service for them maybe a limited uh limited level of requests per i know per minute per hour per day per month uh per week per month or maybe um as they provide you if you more like a premium customer they provide you more you know faster bandwidth so you need to have this ability to configure this and of course it can be built part of the service but why where if each and every service needs to have this piece of functionality and it seems to be redundant to build these lucky beefy services for every time remember like microservices is not about size but it's about a piece of functionality that they need to implement at this piece of functionality needs to be um somehow like solve this particular task but not overwhelm this by infrastructure infrastructure needs to be handled by underlying components another the familiar pattern many of you might might see in the past with using the proxy servers is to log everything and providing some additional um observability points right so all requests that come into um into from your client can be logged and captured somewhere and after that analyzed and you'll have some additional data points for for uh for your application um so let's let me illustrate the concept of um of this by showing the picture of john luke picard uh captain of uh enterprise from uh the next generation and if you would remember this picture for for a moment and uh that's your you know basic service you have a service that does one particular thing however um a lot of things that comes uh into play with this uh infrastructure um infrastructural tasks i would say needs to be implemented either as a part of application somehow or there's a big question like how to implement this and of course many languages in many frameworks providing this piece of functionality and it can easily built into your application but in many cases these days um we don't want to enforce developers to particular language or to particular framework we want to give them choice and that's that's that's why the question comes into play like who will implement all these things right so is it the part of the developers need to plan the part of the framework and after that this stuff needs to be supported what about things that you support this in one language and do you have a parity of the same framework in another language like you have a uh system that developed in java system that develop and go like do you have the same frameworks but uh with uh offloading this to offloading these pieces of functionality to something like api gateway that can i provide you or augment you with some additional pieces of functionality that's what you get um uh in in case of um in case of like particular example uh the way how the cloning solves this problem conch is the api gateway uh and uh it serves this by providing the set of plugins that will enable uh those pieces of functionality where your client will just simply uh will be you know hitting this api in infrastructure will be provided in order to protect your api which brings us to the second point like why you want to do this on the first place so the service connectivity can be um can serve two to two sides one side if you're building the application and this application would be used internally this is one use case in the second case where your application will be served um as externally but in both cases in both cases your application will represent a api as a product so you to develop this uh you're developing something that other people and users will be used um it can be external um users that come to for your for your for a company and for you for the data in your company and this can be some internal so you uh building something building authorization service or you're building service that will be um providing the data like an internal fulfillment service so there's different uh departments might be using this fulfillment services can be internal and can be external and so in this case uh the when the api is offered as a product api gateway can encampulate the common requirements uh that that i already kind of like attached a little bit so those requirements can be uh enforced and implemented with the things that um like the plugins that enable different pieces of functionality and this thing can be standardized so when your organization has a certain standards around what services needs to be used in order to authorize um common users and uh authenticate with different different rules um how uh implement i don't know things around service level agreement and the monetization of those apis those things can be uploaded from your application code into api gateway code another important thing is that it when the api gateway serves um serve a need of building api as a product it also allows you to provide certain level of support for some legacy apis your underlying infrastructure your underlying this part where's my uh where's my pen so uh this part this underlying part can evolve uh independently from client part and api gateway will take a responsibility uh of um supporting a say legacy client if your api changed so there's ability in the api gateway center things provide certain like rules of transforming api calls that old client will issue in order that new api to support so that's uh that's one of the probably one of the biggest use cases and one of the biggest uh you know the selling point why api gateway um serve the purpose so it's not dumb proxy anymore it's not only just like a passing through request um to um underlying service upstream service and the log this somewhere like oh yeah that's client access so now there's more knowledge there's more uh routing so based on um some sort of like a token that comes from the client we can figure out oh let's roll out the um new features we're providing some sort of like a b testing scenario or like cannery deployment and based on certain criteria or a certain percentage of requests this this api gateway will route to two particular will route to particular service or particular version of the service if you want now so since api gateway used in this pattern as a product as an api so developing api and throw them at people it's one thing but you need to support those people and by support i mean you need to provide documentation that explains this api you need to provide the ways how this api can be tested or maybe there are some cases where if you say dual api as a product but for internal needs your backend and your front-end people uh might agree about the api but their timelines on delivering certain features are not in line so developers on the front end need to continue to build their awesome mobile application or or some of the website but maybe api will not be already the implementation of api monitoring so in this case um api gateway can be used to say mock this api and you know provide the ways how your one team can be doing the things while another team will be providing implementation after that when they're ready you just flip the switch and all of a sudden your your clients start using this um mobile application say start talking to a real backend so this is where this is where this api gateway uh becomes a um one of the [Music] connects all the pieces of this puzzle like this is one thing that controls everything so and it becomes more than just like a dump rocks it becomes even more than api gateway for your request transformation it becomes a tool and a platform that supports your api management uh lifecycle so management um management of api uh users and client application and traffic in real time um requires you know i don't want to like oversell this but like it's it's it's a big a bigger part of the strategy of the of the organization and uh the components around um designing and the building apis and uh it's one thing and developers like to focus on this side of things but there are some other things that supporting thing like monitoring providing the way understanding how this api is used it's also part of this um api management uh platform and api gateway become a central piece that connects multiple different phrase phases of designing apis maybe there is a tool that there is a tool there's a tool that allows to design this open api spec that is a more or less common standard of defining the apis and use the spec from point of designing to creating a mock service so the systems in start can start be tested once this design phase is done we're going into the phase where all these day two responsibilities uh we need to be needs to be handled for example on boarding developers external developers to the api so in this case documentation needs to be in place uh things around um like a testable example so that people can start using this api immediately enabling developers of api to use this so like you see how we started this we started this with the very small um uh use case where we have a proxy in between our client and service we gradually go into the point where we um offloaded some of the functionality that comes into particular um like in the gateway side of things and after that we moved the gateway in the center of our rpi management strategy and this is this is how the different platforms um like to position themselves and the things that i just i just talk about it's also available in in quank which is you know can be used as a simple proxy it can be used as a more advanced like api gateway providing these building blocks to enable different features through the plugins and third pieces it also includes some of the things for for for api governance and management and uh providing the manufacturing capabilities uh hopefully by this point you will get a pretty good idea what the hell is this api gateway now and now it's time to go to another pattern it's another architecture pattern and um uh with the many conversations that um i have in the people with people around this many resources that like to describe um like api gateway it is something that routes traffic from north to south and gateway from east west what the hell does this even mean right so where's the south like for me uh like i know that in in america uh people generally very good on figuring out like okay so that's you need to go like a few blocks to north you need to go to a few blocks to south i i don't understand this i i don't have a compass in my uh in my brain so that's why the term when the people talking about this traffic direction uh it's not very that's not very uh sets very well with me uh however i understand usually by north south traffic people mean like inside and outside uh data center and north to east uh oh sorry uh west to to east or east west it's the inside etc but it is not the case because i just showed you that api gateway can be used internally so in this case it also can run the traffic inside the data center and can run the traffic externally from whatever north or south so that's why i don't personally like to use this uh this analogy and this is where the the the um which brings us to uh to mesh which brings us to to mesh idea because like people say yeah like it it's it's routes from east to west like what does even mean now when we talk about mesh we are having multiple different applications so um inside inside our data center let's let's talk about this here for now so inside our data center inside our business functionality now we have different services and those services can uh communicate to each other through gateway you know you see the those the works they already augmented so those services might have a gateway in front of them to some sort of gateway that that provides this piece of functionality around uh authorization identification maybe even traffic encryption and then so far and so on those things are um already exist and we need to you know have a have some centralized management and the way how it was centralized and managed in uh in a world of work it was just a bore cube and in this case the there should be something that would be controlling these things uh and those things can be deployed those board cubes can come from different galaxies in this case your application can be deployed not in one data center it's now decentralized deployment in the way how everything was controlled it's through um war queen in our case in more you know human sense uh the mesh will include uh two components so our borg um our service this guy uh that's that's gonna be our data plane that's where we're gonna be data plane that's something that will be do the work um they will route the traffic providing some of the traffic shaping capabilities providing different um the features and this one it's gonna be our control plane so uh and control plane will be uh will be sending the the configuration information that will store configuration in one place and after that it will ship this configuration to the dead plane uh with the this similar idea in mind with the borg in mind those uh borg bots can be relatively independent so in this case when these uh the the connection to um to control player will lost they still will continue to perform their task same thing with the with the service mesh it usually includes um two components uh control plane and centralized thing that will be providing some of the management capabilities and data plane it's the where something that will deploy it next your application think about this data play as a kind of like a small proxy and small gateway sitting next to your application to reduce the latency and after that it provides a set of capabilities similar to what i just described uh in the previous slide and uh to infinity and beyond we can we can deploy these meshes across data center and provide a unified communication between applications so in this case application developers start thinking not in terms of uh data centers not in terms of servers not in terms of locations but in terms of like meshes and those meshes can be geographically distributed and not only deployed in in systems that support this sidecar model of deployment like kubernetes we have a pod and we have an extra port that will perform this traffic in some systems you just run separate process next to your application um and but it requires some of the kind of automation and some of the tooling around this now so with the with this uh what things what i said so let me uh bring quickly bringing this idea uh to home by uh demonstrating this how your thought process uh should should go about the api gateway so you have um um like use case where you building a service and the service needs to be offered either as api is as a product so there will be some external consumers so the uh that's why we're going if it's going to be used by external consumers you need to make a simple decision if it's been consumed internally you will build a gateway in front of this service in front of this api in a case of external you will have a api gateway that will be sitting in the edge they might have a different requirements in terms of how the services need to communicate internally for example external um api consumers they not be able to get internal authorization system to get access to the system so maybe there would be some third party that also will be deployed close to the edge and if it's internal it can be used with internal api gateway and different policies will be enforced so that's where uh you're going from perspective of like a providing a service that will be used uh by someone as a as a as another service let's uh let's take another route let's take another example so you building this let me erase this guys or i should use different marker so now you're building the something that will be used not anyone else in the system you're building one complex system but it would be another puzzle of your micro service architecture so in this case you're not offering the service as your um as a product so you're not like per se no one will calling except you or your application so um if this system where you deploy support so-called like a side card deployment so you can embed the small proxy process next to your application systems like kubernetes or systems where you can simply run you know two processes your application process and your site card process so in this case there would be a service mesh where you can definitely a benefit for for your use case and the next thing that you need to decide okay am i doing this with existing mesh do i need to care about multi-regional or multi-zone deployment uh it is a part of the bigger mesh or it's just like simple if it's a simple so you just like have a something like a create a new mesh or if it's uh something like bigger in this case the uh existing mesh uh we can join or add another um sub mesh like smaller mesh into the bigger mesh but again like you still can use internal gateway an internal api gateway for use cases where you don't have a luxury of deploying these sidecars to your application maybe it's not feasible from perspective of uh your processes not not super easy to to deploy or maybe there's some of the other limitations but you always will be able to uh use these internal api gateway to um to do the to do the thing so essentially two things that um you probably can come you can probably um get out of here the api gateway admits external traffic and by external it can be edge so this really external or someone that would be outside of the boundary of your organization so we're bringing class traffic from outside and mesh service mesh connects your your application so it allows your system allows the micro services to communicate more efficiently you can untangle the communication between the services through uh different traffic policies and providing the destination and the source of these services more explicit so with this uh i hope you got the got the idea if not i will be available for some enhanced interrogation and for q a but for this my name is victor gamov and as always have a nice day you