SBTB 2023: Denis Magda, Too Far Too Furious: Multi-Region Cloud Applications
Recording: SBTB 2023: Denis Magda, Too Far Too Furious: Multi-Region Cloud Applications
[Music] I don't have any slides for you so don't be disappointed because 30 minutes are not enough to demonstrate how to build uh multi- region applications so that's why I decided why don't we skip the slides I'll show you how to deploy multi- region applications and optimize them for low latency access and then if you find this useful you can always go to internet and search for the content that was created by people that are much more brilliant than I am right I don't want to repeat anybody so the topic for today is tatti furious multi region Cloud applications so my name is Dennis Magda I've been working with distributed databases systems and applications for the last nine years I guess uh before that I was building Java gvm and GDK orical and micr systems but eventually I joined the brutal world of databases and I remained there everything since but that's a very joyful ride so the agenda for today is follows first we need to answer a question why would we even decide and think to build a multi- region application what are the reasons and then we will jump straight to our sample application we are going to we're not going to build from scratch that application does it make sense we have a prototype of a simple application for a pizza company like you take your phone you order a pizza that pizza is being baked for you and then you enjoy this pizza right so we will create we already have some simple microservice and then finally we will wrap up good sounds reasonable so why multi- region applications and speaking about multi- region applications when would you create the one a multi origion application is an app that is deployed across several distant locations and Those Distant locations can be your private data centers or several availability zones in the cloud or multiple regions in the cloud and why would you do this first is high availability why so that because take several regions you have west region in the United States you have east region it can happen that one of those regions goes down if the region goes down it takes down all of your application instances and your database nodes that are being deployed there and you are you have that question do you are you ready to be out for the time while the region is being recovered or you want to get back fast as soon as possible and that's like one of the reasons you want to build multi region applications if you want to be available andate even major incidents that happen that affect an entire region in the cloud the second reason reason is low latency and that's our primary topic for today let's say that you have customers in San Francisco New York in London in Sydney in mumai and they use your application like pizza company and you want them to have the same experience you want them just open this application you want them to be responsive you want the app to not to hang right not to freeze and again to do that you also need to build an up in such a way that the latency is comparable for all of your clients regardless of their physical location and which mean that for in the multi- region context you need to think about global LW latency okay and final reason is data regulatory requirements so sometimes yeah you have this wonderful app that you created but uh if you operate in the European Union or let's say in India then you need to comply with some data regulatory requirements the user data some of the personal data has to be stored in those locations you have to do that and and if how do you do that by creating a multi region application that spans multiple geographies multiple countries so next why multi- region applications the answer this one right uh yeah I just documented this in the wrong place but not a big deal now let's talk about our sample application for the pizza company how we will be moving forward first we have our developer prototype that I will show you and this prototype is going to be running on my laptop and then we will take this prototype and we will deploy it to to the cloud to a public Cloud environment I'm using Google Cloud by default you you are free to use AWS Azure or whatever you like more and that first instance of the application will coincide with the grand opening of our pizza location in New New York City and as you will see when you're talking about global and multi- region applications in relation to the low latency you need to figure out how to store your data because it's extremely easy to deploy an application instance in any region that you choose and you can have multiple instances but if your application instan is deployed in New York City but your data stays and lives in San Francisco then the latency will be high and that's not going to be a fast application so that's why we need to figure out how to deploy your database and then we would our application is going to expand to locations in Berlin and Sydney so we thinking think this way the company keeps growing and it wants to sell the same Pizza in Germany and in Australia why not but we want to have the same experience and uh by looking at these stages of this comp company will experiment with several deployment and design patterns for those multi region applications primarily it will all depend on how you deploy your database and how do you store how you exactly store your data close to your application instances all right let's save this agenda for now and uh that let's start with the developer prototype it should be somewhere under the downloads folder sample applications piz spring Cloud mhm so this application comes with two microservices the first microservice is Kitchen controller you take your mobile phone or you go to website you order pizza and this order goes into the system and this is what the kitchen controller does it accepts your order and hopefully then the CHF will bake this pizza for you so and this how do you interact with this microservice through rest apis we are going to send simple rest apis to this microservice to schedule another Pizza uh for our kitchen the second microservice is called tracker and the tracker is also customer facing micros service what it does once your pizza is already in the queue you want to check the status whether it's still being baked or it's painting or it's being delivered to you okay so two simple microservices we have a Gateway and what this Gateway does it basically going to intercept we are going to send all of those rest requests to this Gateway and if the path URL in this Gateway talks to Kitchen we will redirect this request to the kitchen microservice otherwise we will redirect the request to the tracker micros service sense so now let's deploy the first instance of this application Docker compose up just to show show you how how those apis look like yeah this is a Java application but uh it doesn't matter what's your primary language here it can be any so the point is that the design patterns remain the same whether you use Python Scala Java or C++ when you build a multi region app so looks like it's ready so let's me jump back to my terminal uh that's a pink so one of these HT TP requests is going to yeah it says service unavailable which mean that my microservice is still registering with the Gateway so for some reason it takes time in my Docker environment to register all the microservices but that's my problem anyway so it responded as a database right now I'm using postgress SQL it also runs on my local laptop now let's put the first order so let's say that we want to put the first order the first pizza for New York let's put the second and so here is what's interesting yeah we have this confirmation that the order was placed for New York the status is ordered and this is the latency this is the latency between my backend my application instance this micros service and my database and my database Docker container LS is uh yeah somewhere this pogress pogress instance and when we will be deploying this application to the cloud we will be keeping an eye on this specific latency because we want to minimize this latency and we want to make this latency comparable for all of the client locations where our pizza service operates okay sounds good this is how we post the pizza and you remember that we have their tracker microservice the one that lets us check the status so let's say that I want to check the status for the order number one yep so once it's warmed up no not the post I want to do the get yeah it also comes yeah that's the status we're not going to change the status and that's the latency between the application instance and the database and here is you can see always the total latency but the total latency is subjective it means it depends on how powerful your machine that runs the application back end so it will change the more course you have the lower the latency will be now going to the cloud let's say that the developer prototype is ready it's easy to follow and now we decided to open the first pizza location in New York City and uh here is here is my production environment what I'm going to use I have this virtual machine in the United States east coast and also we are planning we are looking into the future I have two virtual machines in Europe West three region it's in Frankfurt in Germany and I have another machine in Australia close to Sydney because I'm going to sell this pizza in Berlin in New York in in Sydney and I want to make the same latency I want to provide the same user experience now the connection I already have this connection so this is the USA that's my instance in the east coast that's in Germany and this one in Australia imagine right now that we have the first customer who decides to buy this pizza from us the customer takes mobile phone goes to our mobile application and that mobile application connect to this instance right that instance runs uh my uh Pizza back end and eventually what we do this application instance posts the first puts the first order in into the database yeah by the way let me first clean the database I need to start I need to start this application instance I'm using a special script I will just show you what's happening internally while this is happening here let me also start the same application instance I will start the same application instance in Germany and Australia so that we don't uh waste a lot of the time because it will take time to register my microservices with the Gateway all right looks like we are ready in Europe the application instance is running now let's just clean the database yeah it says that the service is still being registered with the Gateway speaking about the database on my local laptop I was using POG SQL which makes sense when I'm deploying across multiple regions I want to have POG SQL that can run across multiple regions I need to have distributed POG SQL as a analytic database pogress is not distributed by default but we have Yuga byb Yuga byb it makes posg distributed it's built on the pogress equal Source score which mean that I take the same application that was written for pogress and I deploy it across multiple regions and I can deploy ugab byb across multiple regions so effectively that's happening my first cluster that I'm going that that that is being used right now it runs across three regions United States East 4 East one and Central which means that and this is a transactional database meaning that my application right now is connected to this node in the East we can easily check it database address us east4 this is the IP address of the database the app is connected to but when I will be putting orders into the system this order needs to be replicated because the replication factor is three meaning that if any of these regions goes down nothing is happening from the availability standpoint I'm not losing any data there are no any data inconsistencies because the database uses draft consensus protocol to replicate changes consistently across your deployment environment for the database so that's why it will take from the right latency perspective there will be the latency will be higher because the database will be replicating all those orders across three regions in the United States Central and East now let's see at the real latencies okay we are putting the first order into the system let's put the second let's put the third Etc so the latency the total latency is like 200 milliseconds I'm running on a low end machines just don't pay too much attention to this total latency it's adjustable but the database latency is 19 milliseconds and this is a good one considering that your data datase runs across three regions Us East one Us East 4 and Central it's quite large area in the United States and this configuration allowed us we pick this configuration because we want to survive outages in the region if any of this regions goes down my application keeps running because the data base is available however let's take a look at the read latency let's say that I want to check their order status for for the first Pizer yeah I'm warming up my connection pools to the database and the latency is formilla seconds because this application instance is connected to this data the closest database node in the United States East one close to New York and the latency is just four milliseconds which is which is good for me I have this multi- region application running and I can have as many application instances running in the United States East sense now our Baseline is 18 milliseconds for brides because we are replicating across multiple regions and 4 milliseconds for reads it's it's a really it's fast application now let's take a look at what's happening in Germany let's say that we decided to sell these pizzas to our friends in Berlin and uh what's the latency right now because this application instance is also connected to the same database note in the United stes stes let's try to put the first order I'm putting let's say order number 10 and I'm putting this order for Bing and while we are doing this let me also do the same for Australia I want to put an order for Australia in Sydney someone from Sydney ordered the pizza all right let's put a few orders so here is right now take a look at Burling the latency is 191 milliseconds it's 10 times right longer where is this one this one it was here as it was around 18 milliseconds anyway why why this happening because the application instance needs to send this request through the land under the Atlantic Ocean before it gets to the database note in the United States that's why you have this much higher latency if you take a look at the latency from Sydney it's even two times worse than for those who live in Germany because you either need to travel through the Pacific Ocean or you need to travel through the continents before through Atlantic Ocean until you get to the United States and this is the problem and this is the problem that we want to tackle because let's say that the companies wants to expand successfully in Europe and Australia but but the latency is really bad and right now I'm just sending you know just some a few requests like one request a minute but what happens if I'm going to generate let's say hundreds of requests every second that's that's going to be a big deal one of the options is and the most trivial one is let's say in the beginning when you scale most of the time what we do as users before we buy any pizza we serve we browse right we take this application we take a look at the pizza catalog or probably we will be building our custom Pizza which mean that we are reading we are sending read requests to our database and the First Natural Choice might be all right let's optimize reads there are many ways to optimize read request one of them is just to deploy read replica notes of your database to the locations of your clients and let's do this so we already have this primary cluster of the database ndes in the United States that keeps consistent copy of the data and survives outages but for my clients in Germany and Sydney I want to to use read replicas so let me restart their uh microservice instances they will be reconnecting to a different database noes I will show you in a moment what those noes are so here is this cluster the the main cluster and down below I have two repca nodes one of the replica nodes in Australia and the second one is in Frankfurt so when comes to replica nodes the changes from the primary database cluster are sent asynchronously to the replica nodes meaning that when you will be reading from the replica you will not you might not see the latest data but what's important as long as this is a transactional database even if you don't see the latest data the data is always consistent so you will not have any inconsistencies like all right like if you update in multiple tables we in a single single transaction then all of those multiple updates will be visible to you at once on the replica node it will not be like one table a is updated but changes for table B are not delivered yet that's not happening so let's see how replica nodes are helping us so here is right now just a double check yes this application instance is connected to replica in Europe uh this one is to Australia how about we send uh no not the send let's just get the order status for order number 10 and here is order what was that order number 20 we don't need the location field uh hold on sorry sorry I messed up a little bit and here Order ID yeah the first time when you connect through the replica note what's happening why do we have this delay the database the the application instance opens connection pools database ction pools through the replica and the replica needs to load some metadata from the primary database cluster so which mean that you're waiting while that metadata from the primary database cluster is loaded so that's why you usually warm up your connection pools when you restart your application instance so uh if you repeat this several times uh let's see the latency 3 Mill seconds looks comparable right to the United States this is how fast it will take to get the state status to complete ration in Berlin you get you have the same latting if you do the same for uh Sydney let's run this a few times it's also for milliseconds so the first job is done you have what we have let's do a quick summary you have multiple application instances running in the United States Berlin and Sydney for your pizza company and right now we achieved the same low latency for read request so whenever your customer opens your mobile application web frend starts search searching through the pizza catalog the latency will be comparable no matter where where customer lives but the second problem let's say that your company is getting bigger and right now they want to streamline all those operations which mean that they want to minimize the latency for right requests as well they want rightly requests to be as fast as possible across all of those locations and read replica notes are not helpful here why because read replica note will basically forward all those requests to the primary database that is in the United States that will take a lot of the time however for that we are going to use another database cluster called geop Partition so let me restart my application instances one last time and then I will show you the latencies and we will take a look at their database cluster okay G or this one connects to Frankfurt and uh this one connects to Sydney so what this geop partition cluster does it's here I have nine database nodes and I'm using this time I'm using that's a single stretch cluster one database instance that's stores data in the United States Europe and Sydney and in every region so here is just for the sake of experiment I decided just in every country I decided to select one region but you can have as many regions as you like in every country but within every region we have we use three availability zones so with this configuration with in every country you will be able to tolerate Zone level outage and the replication factor is still three meaning that if you put an o pizza order in Sydney then three copies of this order will be stored across three availability zones that's for the availability from the from the availability standpoint and then I have let's say database notes in Frankfurt Virginia in the United States from the table standpoint like what this cluster is capable of it will be taking your orders and it will decide the database will pin store this order in one of the locations if you ordering pizza from New York then this order will be stored in New York New York if you're ordering pizza in Sydney this order will be stored in Sydney not in Europe and that's a trick this is how you can minimize this latency so the database does this automatically for you from the application standpoint what you do in your table in your pizza order table you already have this column city name and then you can create so-called partitions in pogress and let's say this pizza order USA partition is stored in the United States this one is in Europe when your application does this insert let me show you here kitchen you're still doing this insert into the pizza order you provide the city name location that's all you do internally the database decides where this the order is going to leave all right so let's uh quickly check the latencies first let me just delete everything from that geop partition database instance so like this delete operation what's what's also interesting here every every um application instance is connected to it to to to each to its own database nodes this one is connected to the nodes in the west uh this one is connected to the nodes in Australia and this one is connected to the nodes in the United States but I can use this single database connection to run global requests it's like if I am ready to take that higher latency because probably I need to delete everything or I'm traveling to Europe and I need to order pizza from my family that lives in New York I from the application standpoint you don't need to do anything just your application is connected to this database nodes and if this database no decides all right this order actually needs to be transferred to the United States the database will do this you don't need to do that all right let's do the latency test let's okay we are putting again the first order for New York uh let's do one two three caches roed up so the latency this time is like 8 millisecond 7 milliseconds it's slower because right now I'm I'm using a single region for the United States and the data is replicated across three availability zones it's obviously faster so you can do this configuration 7 milliseconds for the United States how about uh Germany how about customers from Berlin let's say that we are ordering from Berlin and let's do the same for Australia so here is the same in geop Partition cluster there is a concept of the primary cluster like there is some primary metadata and this primary metadata is stored in the United States which mean that when you open your database connections from the application the first time this metadata is also needs to be transferred that's why you need to warm up your connection pools but once once it's warmed up let's start putting all those orders for Berlin does it look familiar 7 milliseconds from Berlin 7 milliseconds from the United States the same latency right the same application just a little bit different database configuration so let's repeat this exercise for those who live in Australia 7 milliseconds sometimes 9 milliseconds all right so it's just the same single digit low latency millisecond latency and uh if you want to check the status let's do get request let's read the status order number one for New York order number two it's like 5 milliseconds 3 milliseconds okay so let's do here we want to read order number 10 for burin and that that's another microservice and here is again right now we are opening this connection pool from the second tracker microservice and it loads it's loading metadata from United States it happens only once when you restarted your application and you need to warm up your caches but once it's loaded you will see that the reads are going to be fast all the times so yeah 3 milliseconds for burin yeah it it takes much more time to load this to Australia from the United States but prohibitively long time what's happening right yeah we have minute left and hopefully this query will will end sooner let's do this once again yeah the metadata was loaded is the same time so generally this is how you can build multi- region applications that SC scale by designing your application from top to the down select Cloud regions that you think suit the best your user locations then provision your application instances in all of those Cloud regions and decide about your database deployment you have multiple database deployment options it can be just single cluster in one region across multiple availability zones if you're if you're all primary users let's say live in San Francisco if you operator across the United States you can provision across several regions and you can optimize reads if you have customers globally and you want to optimize just reads you can use read replicas or if you want to have low latency for both read and write requests across distant locations then you have geop partition cluster so keep this in mind that's going to be an excellent kind of reference book for you when you will be building multi- region applications across locations thanks for coming and have fun so we for the questions where should we [Music] head