Building scalable distributed systems powered by Dapr OSS.

SBTB 2023: Kendall Roden, Building scalable distributed systems powered by Dapr OSS.

Recording: SBTB 2023: Kendall Roden, Building scalable distributed systems powered by Dapr OSS.

uh so today we're going to get quick and dirty into the slides and then we're going to jump into demos for the rest of the time we only have 30 minutes together so I just want to make sure we cover as much as we can and really talk about the Dapper open source project uh how you can get up and running with it how enables developers uh so I'm curious how many people have heard of the Dapper project before wow wow okay this is super exciting like completely Green Field we love this so I'm going to go from zero to 100 and hopefully at the end of today 30 minutes from now you'll walk away feeling very confident in what Dapper is and how it can enable you so curious how many of you are actually back in Engineers developers writing code most of the time perfect I think you'll benefit really well from this then so uh what can you see can you see the you can see a slide right the intro okay perfect um my name's Kendall rhen by the way I uh am a product manager at a company called diagrid and we basically build products to enable people to use the Dapper open source project at scale on top of kubernetes and then we recently released a new product that essentially allows you to use um a set of open- source apis to build distributed applications regardless of Hosting platform so no longer constraining Dapper to running on top of kubernetes but now you could use other uh SAS or pa-based services in you know the primary Cloud providers so at one point or another probably in the past several years you've all had some level trying to make sure I'm not screaming at you guys um how many of you have ever been part of like the cloud native hosting conversation right like you've heard the conversation around iaz and PAs and should we go serverless and how much do we want to abstract away from the developers and how much infrastructure do we want to manage like is that a pretty familiar conversation I would say uh yeah most of us have have probably gotten in at least one or two debates on what cloud native really means and does it have to be microservices in kubernetes um but essentially one thing that we find that's missing from a lot of these conversations is where developers fit into this narrative right how do we create and enable developers um Beyond just shoving things in a container and running them on kubernetes right it provides us a level of infrastructure portability across Cloud providers um and obviously on premises as well but we don't necessarily just have portable code right our code still has dependencies libraries um reaching out to specific services and that's not necessarily something that translates from cloud Prov to cloud provider even environment to environment so that's really where Dapper was created and comes into play is by creating and enabling a consistent set of apis and an infrastructure layer or excuse me an abstraction layer through which developers can code against infrastructure without bringing the details of that infrastructure into the application code so having a way to interact with consistently infrastructure providers without that SDK logic ever having to come into contact with your business logic and this is really great because it does provide this new level of portability at the application layer so we're no longer just you know taking all the dependencies wrapping them up in a portable package but we're actually taking code and making that code portable so that we don't have to rewrite that logic whenever we move from Pro uh from cloud provider or service provider um you know from one to another or even from the local development workstation where we're leveraging developer Centric services and then moving toward a more production uh ready scenario where we're using you know maybe kofka instead of something running locally on redus on our local machine so that's what Dapper is the distributed application runtime are are any of you familiar with the cloud native Computing Foundation is that a pretty familiar uh entity okay awesome so Dapper is an incubating project in the uh cncf so that's where you know projects like kubernetes have been incubated and then successfully launched into graduation uh so a lot of Open Source projects tend to gravitate toward finding uh you know a place in the cncf because it provides a license that's guaranteed long term so there will never be like a paid uh you know version of Dapper that you have to subscribe to right you can have consistency and confidence that the project will always be open source and free to use so with the distributed application runtime when we're looking at the actual architectural model of how this runs how you can use it um it runs as a sidecar so if you're familiar with kubernetes do we have any kubernetes you know software engineers in the house infrastructure Okay cool so if you're running on kubernetes the way that this looks is that you'll basically deploy a pod which hosts that container um with your application code and then you'll also have a side car uh so this is probably a familiar uh Paradigm if you're familiar with like the service mesh primitive uh where you have a single side car that runs alongside each application and that's how Dapper operates as well and then it provides a consistent API layer so through HTTP or grpc you can communicate from your application across Local Host to the uh Dapper sidecar and access a set of distributed application building blocks so um if you've ever built distributed applications you've probably written a lot of code to handle things like publish subscribe communication how do I do secure service invocation with mtls how do I do State Management right I want my applications to be able to securely access and retrieve State um I also want to do things like um maybe even distributed actors running State flow workloads at scale or even Implement workflows so if you've heard of like temporal right they provide a workflow engine Dapper actually has an API that codifies the uh workflow basically management um process and so I'll show you a little bit of what that looks likey uh but yeah we basically provide a lot of these patterns out of the box as apis that you can consume and then on top of that you get a set of cross cutting concerns that are baked into the apis so things like resiliency right I no longer have to code you know for me I'm most primarily a net developer I don't have to write any poly logic to basically say I want to create a circuit breaker or I want to you know retry this particular call or I want a particular timeout the Dapper apis provide that to you in a declarative sense and then you can also set like security Paradigm so we have like API token validation app token validation um things like Scopes which allow you to configure access to backend um uh infrastructure providers and ultimately most customers run this on kubernetes and it's completely application code agnostic so regardless of code or framework you can call these apis so if we're looking at it in practice making a little bit more practical and taking those building blocks and turning them into apis think of things like V1 invoke this would be the way that you would call out to other uh Services running within the kuber cluster or V1 State uh so take out all of that redis logic take out all AWS S3 logic and instead you're going to call out to the V1 State API and the Dapper side car will broker and facilitate a connection to whatever backend infrastructure you choose um and then the same goes for the other apis very consistent uh have created like a a very clear coding model around these things V1 publish V1 secrets for Secrets retrieval all of which really provide that abstraction layer to uh to make sure your developers aren't bogged down by infrastructure choices that are made by the platform team so this is kind of a picture of all of this together right think of an actual microservice based application or even mod modular monolith that has a variety of different functions it needs to perform like retrieving Secrets or maybe even being invoked based on an event that comes into um a particular database or make uh you know a publish to a broker right all of these would traditionally be done by creating direct calls to those backend Services wiring the authentication mechanism um but with Dapper what you get is that consistent layer with all of the Dapper side cars and then you get that abstracted API call so now it doesn't really matter what broker I'm using or which database I'm using instead I focus on creating really clean reliable code and then the platform team can provide me with you know whatever backend infrastructure based on either the environment or uh the changing requirements that that application has or even we all know in the cloud space a new service that's created that might be a better fit for what we're trying to to accomplish so how does this actually work on the back end so when we talk about the infrastructure swapability layer how do we Implement that um and the way that's implemented is through a series of what are called Dapper components so I essentially tell the Dapper side car hey when I call out to the V1 State API use this particular component or when I call out to the pubsub API use this particular component um so these are all named uh kubernetes resources that your sidecar can pick up depending on the URL of the API call it will know which component to use so there are over like I think 135 different components supported today across all of the different apis um which once again is is really great and because it's completely vendor neutral and Community Driven we see new components being added every day as new Services emerge and especially even the new kind of upspring of serverless offerings that facade some of the the backend infrastructure even those uh can plug and and and and prove as a adaper component very useful so are you all excited we're going to do only Dem most from from now on uh so we got through those pretty quick uh so this is what I'm going to show you so I thought it was good to kind of just set up how Dapper Works locally and then all kind of progress into kubernetes but essentially the only requirements for getting up and running with uh the distributed application runtime or Dapper is doing you know you're going to install the CLI like you would almost any any tool and then you're going to run a Dapper and knit and what this Dapper and knit command will do is essentially fire up and start running Dapper on your local machine so it's going to pull down container images um you can run it without Docker or without like a pod man um you can run it in slim mode but what's really nice is what this provides is a default uh Pub sub broker and a default state store component so it'll use Docker it'll run redus and it'll basically wire up the side card to automatically use that local redus instance so you can get up and running with the apis and then obviously swap out that component without changing the application code and then the other thing you're going to do is run adaper run so this is really the command that's going to launch your application and the sidecar process so you don't have to be running on kubernetes locally you can just be running you know your application process and then we'll launch that sidecar process which is Dapper and then ultimately when you deploy to kubernetes these would be running in the same pod as containers um but what's nice is the Dapper sidecar is basically going to introspect a file that we pass and that file has a set of dapper resources in it so this is going to be all the components we want it to load which are going to inform the API calls and then other cross cutting concerns things like I wanted create a new Dapper Pub sub subscription so I can programmatically or declaratively subscribe to one of these components without changing the application code should I choose and then we also have like resiliency policies for example so it's going to pick up the resiliency policies I apply and Target particular components um or applications uh whenever I'm making those calls so this is what you're going to see locally so we're at 12:36 I think I have how much time do I have somebody keep me honest I've been talking for 10 minutes we got 20 minutes left okay okay we're doing good um are there any questions at this point that anyone wants to ask is it adding up is is the math mathing hopefully okay awesome okay so we're going to dive directly into visual studio and is the audio still okay I feel really loud but okay as long as y'all are y'all can hear me and it's not too much okay so I'm dropped into Visual Studio code don't get too you know too bogged on with any of the details here is it zoomed in enough you know when you zoom in a little bit more sometimes it gets hard to see maybe a little bit okay I'll do a little bit more but you know it might make it a little bit less legible but that's okay you know we we work with what we have so the first thing I'm going to do like I said is run my Dapper init command I've already installed the CLI and because I already have Dapper initialized on on my local machine it's basically going to tell me like hey you already have the latest ver version of dapper installed like you're good to go you don't need to do anything else so if I didn't have this already what this initialization process will do is the first thing it's going to do is create a Dapper folder for me with those components so let me show you a couple things so one is I'm just going to CD into uh the directory that's created by the initialization process and what you're going to see here is I have this components um you know folder or directory so if I CD in here and I take a look I can see I have a pubsub DOL and a state st.l so these are going to be those manifests that inform the API by default use reddis to Target whenever you're making API calls so if we open these up and just take a quick look at them I've moved them into my local directory to modify them um but this is essentially what a component looks like right it's a a kubernetes manifest step manifest definition essentially and it has a name so the name is really critical that name becomes the unique identifier for this particular component and then it has a type so remember when we talked about like the swapability aspect this is the only place where you're going to have any reference to the backend infrastructure there will be no code reference at all in your application so this could be Pub sub. Rus pubs sub. AWS SN ssqs it could be pubs sub. aure dos servicebus right the list goes on there's a ton of supported components and then based on the component type you'll have a series of configuration values you can pass in things consumer IDs right for Kafka uh so there's going to be different metadata depending on the backend infrastructure that you're targeting and you can see out of the box it's just calling directly to the redus Run Local Host which happens to be running on top of Docker in a container and that's why the state store and Pub sub look the same right because redest streams is a pub sub provider and then you have Reedus the state store instance so I've moved these over into my local um you know my local directory where my code is sitting and that's really where I want to get into kind of the nitty-gritty of let me see if I can get back here okay cool so now we're now we're in dropped into Visual Studio we have some components set up which I moved into this local Dapper resources folder but I really just want to show you what the code looks like first and give you a taste of what it looks like to make use of the dappery apis so I have two applications both are written in net but like don't let that constrain you I just didn't want to overwhelm by adding a bunch of polyot Ling which is in a 20-minute demo um but essentially we have one consumer and one producer right so the objective of these applications the producer is going to publish a message to a pub sub broker and the consumer is going to receive that message and store it in a state store right so pretty basic um you know setup here so if we take a look at the actual program it's super simple and we'll take these break points out in this search just to to make it a little bit cleaner so this is really the few things that you want to note so one is I'm using the Dapper STK so obviously the STK provides a wrapper around those V1 State V1 workflow uh apis and just makes it a little bit friendlier to use but I can call them directly using HTTP G or grpc I don't have to use the SDK uh but the Dapper client will essentially start up and the Dapper client's what I use in order to communicate to the Dapper side side car process that will be running alongside my application and this is the only line of code outside of the client instantiation that I have to create to publish the message can we just clap for that that's pretty impressive if you've ever published to a broker before yeah yeah it's kind of cool I mean your first thought's going to be like it can't be that easy my challenge to you would be like try it out because it really is um but yeah essentially we're going to use the Dapper client and we're going to publish an event and the two pieces of information we pass in is what is the name of the pub sub U component and then what is the topic on that component I want to publish to so if you remember we had a component for reddis so this is saying Hey I want you to go find that pubsub component named pubsub and use whatever that backend infrastructure resource is to publish this message and do it on the orders topic passing in an order payload so it's really is that simple and then we have a second application that's going to be running as well and this is the consumer application so similarly super basic right I have a sing single line of code that's receiving that order that comes in and it's basically going to take that and store it in store so if you remember we had that Redd state store running locally it'll use that out of the box because I called that component state store now I can switch out what that state store is which I'm going to show you as well um but super basic code and there's sdks for I think seven different languages we have rust python go uh Java so like it's very you know language agnostic super friendly in that regard okay awesome so the only other thing that I want to show you is this component manifest right uh so we we already looked at the pub sub rdus component so you can see here it's called Pub sub Rus we have Pub sub. Rus as the type and then we also have Scopes so Scopes are A Primitive which essentially allow you to dictate which applications can access a given infrastructure component um so that's just another kind of added security measure there and then we also have the state store which once again is going to point to that redus instance and then there's a couple of things that I added uh that I wanted to highlight and one of them is a declarative subscription so how do I actually make sure that the consumer receives the messages that were published using the uh the pub sub API and I can do that using either a programmatic subscription or a declarative subscription so in this case I'm using a declarative subscription and I'm going to say hey this is called the pub sub- rdus subscription it's going to listen on the orders topic on Pub sub- Rus right like that component that we already specified for redus I want to listen to the orders topic there and anytime a new message gets delivered I want you to send it to this particular route on the consumer application so like pretty straightforward once again really nice you don't have to change your code in order to do that so what I want us to do is I want us to actually wire up these applications side by side so there's multiple ways I could do this one is I can run each individual application with a single command and launch its process I can use like my debugger and then just run the side car to connect to the debugger or what I can do is I can use a dapper multi-a run file so uh if you've ever played with Docker it's kind of similar to Docker compos except there's no container dependencies right uh the only objective is hey I want to run two applications one called producer and one consumer I'm going to run them on these particular ports um and I want you to launch the Dapper side car alongside these applications so I'm just going to do a Dapper run and that should wire this up for me and it'll send all of the logs to the same um the same console so we'll do first we need to get into that directory where the application lives and we'll do a Dapper run and pass in that file oh I ran a Dapper run run we love that very eager um but here we should see cool so I'm going to zoom out a little bit just because it's a little unfriendly um but what we what I want to call out is a couple things so you're like whoa that's a lot of logs I have it on debug so don't panic um but there are some really good things to to call out whenever we do start these up okay so one of them is going to be the first thing that we see is we're starting a a Dapper with the app ID of producer right so this is going to launch the sidecar process for that first app and what it's going to go through and do it's going to start loading some of these configuration um resources that I've that I've sent into it right it's loading resiliency policies it's loading a local file Secret store so I can actually retrieve Secrets as well using the Dapper API and when I'm running locally I can get it from a local file and when I'm running in kubernetes I could do kubernetes secret or Hashi cor Vault you know once again list goes on and on uh it also found a component for pubsub AWS and pubsub AWS is actually going to be uh the component that I swap out the code to use right now we're using that pubsub uh- rdus and we can see it found the pub sub- Rus component so all of this looks good so the the goal is really just to make sure the components we expected to get launched got launched and then once again we'll see application logs as well so we can see that our app is up and running on 5200 and then the consumer is up and running on 5300 so everything should be pretty good at this point what I need to do is make sure that I pass in the correct environment variable here so let's take a look at this Dapper run so one thing I didn't do and this is something good for me to not right is I need to pass in the environment variable that tells it which Pub sub broker to use so if you remember I'm going to use the redus one so I I think right now it's hardcoded to use one called pubsub but what I'm going to do is in my producer I'm going to add one more environment variable here called pubsub name and we're going to pass in the name of this pubsub redus component cuz this is the one we want it to use so I forgot to add that in so we'll do that perfect all right so I'm going to rerun these just so it picks up the uh the new environment variable that I want it to pass in and then how about we go ahead and test these out so what I'm going to do is I'm just going to post a quick local um command here I'm going to I'm going to do a quick post to the orders method on the publisher and then ultimately we want to see the subscriber receive that message so I'm going to send this request and that's exactly what happened right we see it success the uh producer successfully published the order order number 12 to the topic orders and then we see that the subscriber picked that up and successfully persisted that order message in the state store so once again we didn't see any crazy code any reddis inside of our actual application but if we take a look I'm going to pull up redus Explorer probably should have here we go and what this is going to allow us to do is just see that it actually did call out to that local reddis instance running ins of my uh Docker container so if we pull this up here and I probably should have cleared this out should have flushed all the keys but if we look at orders we can see the stream um so we can see that the message was published right through reddish streams and then we can also see if we go to let's see if we go to the consumer we can see that the uh the consumer stored a key of 12 and the data inside of that was the order uh the order ID 12 so not only did the message get success delivered through the pub sub broker but it also got persisted within the red estate store so what's cool is uh how would youall feel if I just like swwa this out and we use AWS instead and I don't change the application code nice don't get too excited okay so all I'm going to do this time is I'm actually going to change that same environment variable so maybe it was good that I showed you all this before so instead of using pubsub rdus we're going to use pubsub AWS okay and if we take a look at the Dapper resources we can see a little bit more complexity in this particular component so we're using the SNS uh component for AWS so that's their simple notification service simple Q service and then we're also passing in some secret values from a local file so my goal here was to abstract away uh the authentication credentials so that you know you wouldn't see it as an audience however what's nice about this is I can swap out the Secret store and that Secret store could point to kubernetes or uh you know Azure key Vault the list goes on and if we look here it's actually using another component so if we look at Secret store it's using a file that's located here the secrets. Json um and it's basically picking up and loading those secrets and then once again I can swap out that implementation for something more uh production ready whenever the time comes so all we did was swap out pubsub do- AWS and we're going to run the application again so no code changes no nothing uh we've gone from my local machine we're deployed on top of an eks cluster let's say and we're going to call out to um SNS sqs so I'm going to make the exact same call and what we can see here is it successfully published order 12 to topic orders using uh AWS so I think I may have the um I may have the the broker up just so youall can see so you can see here it's actually making use of this topic and what's nice is you can actually give it permissions to go in and create uh dynamically the topics and create dynamically some of these resources so if this topic didn't already exist it would go out and actually create that topic on my behalf or in the metadata config I can say hey I don't want you to have permission to basically autogenerate resources within the um the broker but yeah so very simple switch no code changes at all and I'm using a completely different infrastructure provider which is pretty compelling there's really nothing else else out there that lets you use something you know like this today and then how are we doing on time we got five more minutes cool I got more for you are you ready to see it are you excited how we doing okay cool I hope y'all are liking the the more demo forward I know 30 minutes isn't isn't too much time but okay wanted to show you a couple of other things so one is I've changed out my local uh actually I could have I could have done this right now so if if you remember I changed out the pub sub to use AWS but I'm still using the state store running in redus right I didn't change out that component so let's say that I and I don't even know if I actually practiced this one today so fingers cross it works and I got everything set up but hypothetically speaking if I do a d a Docker PS I can see that I have this like redus um this Rus container running right it's called Dapper redus sorry that uh that zoom out really is is fun stuff but we can see we have Dapper redus running I'm going to stop redus um so what that's going to entail is it's going to kill the state store and then we're basically going to see if the calls will retry due to the resiliency policy that I've created so and the one thing I do want to do I want to remove this component um from the pub sub provider otherwise this this won't work out give me just one second okay so we're going to run this one more time perfect so what we should see is we're going to go do a Docker stop and we're going to kill Dapper rdus so what happened is oh no dang it okay this loaded that component one second the publisher starts yelling which is actually pretty good but we don't want it to yell in this case let me just remove this PBS up provider okay we love a good live demo um it's always great okay we're going to do this one more time and then we're going to kill this again okay fingers crossed okay so we're using the AWS Pub sub we're using redus as a state store I'm going to publish a message and what we should see is that the state store obviously struggles to save it in reddis because um you know it's not available so we'll see if this uh consumer application if everything's up and running let's see am I even running it consumer what did I miss anybody know I know what I did it's the subscriber right remember I had my subscription pointing to a different component so I need to switch out the the uh subscription so that's what I'm going to do now still got a couple minutes no sweat off my back we've done this before so we're going to go into the subscription and what we're going to do is point this at sub AWS right that's pretty critical because otherwise it's subscribing to a component that no longer exists so we'll try this one more time fingers crossed everybody cheer for me get excited okay and then I have to start this because otherwise it's not going to start up right if the the initialization process will fail if the state store is not available at startup so what we really need to do is for it to be healthy and then for it to go down all right third time is a charm I have heard has anybody else heard that saying yeah okay so we see all of the old subscriptions coming in probably should have flushed that out that's that's good so this is picking up all of the old um published messages that I put on top of the um topic right it's actually reading off that topic now well let's go kill uh Docker the do Docker uh state store and what we should see is it's going to start having issues right so now you can see okay this isn't working very well we're having issues and then we can see it's retrying it's retrying and then now the circuit breakers is open so it's essentially going to continue to retry until that state store comes back up because of this resiliency policy so you can go in you can set you know retry policies timeouts and you can Target particular components like the state store which is why this is running and if I was to go and you know revive that state store and think of this as being you know really good for transient errors especially when you're running in a cloud provider um you can see that it's going to recover and start persisting again which is really nice without any of that retry logic being hardcoded in so I have one more minute which makes me really bummed because I wanted to kind of show you what it looks like running on kubernetes but it's pretty simple right I did a Dapper and knit but instead of initializing it on my local machine I passed in the kubernetes flag and so if I just do a a k get and S we can see where we are uh all of the control plane gets uh deployed to the Dapper system and then I basically Al have another name space uh which is the uh the order app which is where all of the code is running so if I get my pods I see here that there's two pods running both of these have two containers the apps the app and the sidecar so I would see those sidecar logs it would load my components if I do K get components I can see I passed in the pubsub a AWS in the state store um and once again I could swap these out I'm using kubernetes secrets on kubernetes and the last thing I wanted to do um just before I round it out is show you kind of what Dapper provides from a visibility perspective so this is one of the the products that I work on at diagrid but you can see here I can actually get app visualization between two apps so when I start making calls I can actually isolate in I can see all of the different components that are running I can do dapper upgrades um so this is just a really great way to get visibility very similar to the open source uh Dapper dashboard probably should have sent some requests in to be able to see that at map visualized but um yeah ultimately this just gives you really good taste of all of the Prometheus metrics um the tracing that's available and even out of the box um I get Zipkin so if I take a look here this is running out of the box on my local machine when I did that Dapper in it it goes ahead and sets up tracing for me so I get tracing out of the box end to end and you can plug that in with basically any tracing provider so uh thank you all so much for your time it's been a pleasure I know I'm I know I'm coming up on the end I tried to pack in as much as I could if you're interested in learning more about Dapper I would really encourage you to get involved in our Discord Community um we're always looking for contributors um and just people who are interested and want to learn more about the project so interested in Dapper reach out to to us at Dapper and then you know if you're interested in working with us at diagrid please feel free to reach out to me and connect directly so thanks for your time really appreciate it yeah hope you enjoy the rest of the conference