Scale By The Bay 2020: Nolan Grace, Cloudflow: Spark, Flink, and Akka Working together on Kubernetes
[Music] again my name is nolan grace so i'm a senior solution architect at lightbend um please follow me on on twitter and if you're trying to look at any of the code that i i show or talk about um throughout this session you can find it on github so um being a solution architect at lightbend has given me a really great lens to both work on some of our open source projects at lightbend as well as talk with a huge range of organizations from small companies to big companies and see what they're doing in you know the scala slash reactive ecosystem and so i've been able to get a really um you know interesting lens into the landscape of what's happening with a lot of technology right now and what are a lot of different organizations challenges so you know with that said um not everyone is aware of light bend i think in this community a good number of people are but just kind of a quick uh introduction lightbend is the organization you know the commercial entity behind scala the programming language as well as uh you know the you know main commercial organization behind a lot of open source projects such as um akka play and lagum but specifically today we're going to be talking about cloud flow which is light benz you know stream you know fast data streaming tool uh running on top of kubernetes so why did we decide to build this new tool well um you know from my experience just talking to organizations and and seeing how people are going about building streaming architecture um there was a big glaring problem or one big glaring problem that uh occurred to me was this you know monolithic nature of how people were building uh apache spark or apache flink applications a lot of organizations will certify apache spark or flink as a tool that can be used and then all of a sudden that becomes the only tool to solve streaming problems in in the toolbox and so you know people are just using apache spark for every single streaming problem and you know the the nature of how you build spark applications and deploy spark applications people started to build them very similarly to how you would build you know code on a cobalt mainframe and that is this giant monolithic hunk of code that needs to be deployed as this giant application um and and it really wasn't taking advantage of a lot of the microservice best practices that we've um that we've developed over the last you know 10 to 15 years beyond that you know once organizations were able to move past that piece and realize okay well i should be you know this this streaming you know when i start building streaming applications we should break them down into smaller independent applications take advantage of um microservice best practices where they're necessary and um you know this started to cause some problems so lightbend had a product called the fast data platform we ran into a lot of projects that had this type of topology where all of a sudden you have a large range of microservices and in between each of those microservices you're going to use some sort of data backplane like kafka some of the micro services might run on kubernetes or straight up aws instances spark may be running on dc os or mesos and you know there so there's all of these problems that are now no longer associated with application business logic but much more focused on plumbing and you know application topology so now your developers are much more focused or spending a lot more of their time worrying about connecting things together as opposed to building you know services that better serve your users and so this is another huge issue with building complex streaming applications and these are really the reasons you know the things that we were seeing in the industry which made us want to build this open source project cloudflow so before we get into all of the specifics of cloudflow itself i want to set up the groundwork so there's a lot of technologies in cloudflow and um you know everyone in this group you know um watching this session may be familiar with uh most of these but uh just to uh to make sure we're all on the same page do a quick summary of kind of all of these tools that are involved with building cloudflow applications so first we've got apache flink and spark and so i'm going to group these together for the sake of time and i'm well aware that there's a lot of big differences between these tools but they're more closely related than than some of the other ones so apache spark and flink are are both exceptional tools for processing streaming data they have different ways of of going about processing that data but they're really good at very specific operations specifically windowing and aggregation taking a ton of data from a pipe like kafka or something like that and being able to move all that data around nodes be able to bend and twist it this is where both apache spark and flink really thrive and to help people understand windowing and aggregation as i mentioned we have we have messages that are coming in on one side um at some speed and we want to look at you know a specific window of data for the last hour i want information about you know i want to group data by user id and location i have the beginning time stamp and i want to sum over that hour of how much money people spent at these specific locations in this way you can use this type of information for a lot of different things potentially we'll be talking about fraud analytics in a little bit so you can be able to say well you know if there were 15 transactions at a you know walgreens that's really far away from where you last used your card maybe we can use that information to better indicate some sort of fraud next we have akka so akka i'm going to talk about the three main kind of pieces of akka and that is aka actors aka streams and akka clustering and so the the core element of akka is something called an akka actor so an akka actor you can think of it similarly to how you would think of like a java object or something like that you you know initialize this aka actor and rather than interacting with this actor using something like a getter or a setter you're going to use asynchronous message passing so every actor has a mailbox and so what you do is you asynchronously send a message to the actor's mailbox and then that actor single threaded processes those messages in order so an actor is just going to take something out of its mailbox it's going to do something with that message and then it's going to take the next one out of its mailbox and so in this case right here i've initialized two actors and i send a message to the first actor that message that actor processes that message and maybe that message says hey say hello to your neighbor and so that actor then sends another message to a different actor and then the the second actor responds with another message so this is how actors work actors are designed to um you know be to to manage state there's supposed to be state or information inside of these actors and um you know then you can start modeling out really complex things like shopping carts or users or things like that inside of these actors and you have a really powerful abstraction for parallelizing these workloads built on top of aqua actors is something called akka streams and akka streams as compared to apache spark or flink is an extremely lightweight um streaming protocol so akka streams is really not designed from it's it's designed from the you know ground up to be this back pressured reactive stream so the goal is that you know we have this source maybe we're reading information from kafka we've got a flow we're filtering out um maybe bad data and then we've got a sync where maybe we're sending a rest api call to some other service and so in a back pressured stream rather than having having this push based model where data is being pulled from kafka and then pushed downstream and in that case you have to worry about you know maybe this flow stage is really slow a lot slower than the source and then we have to worry about you know how do we buffer things in between here how do we make sure that um you know lower or slower downstream operators aren't overwhelmed in the case of a back pressured stream it's all pull-based so your downstream operators when they have you know cpu cycles to do work when they're available then um they send a you know they request upstream hey give me more work and then the source will provide more work and that you know using this model you have a really efficient way of distributing cpu or memory resources and you can make sure that no downstream operator will ever be overwhelmed and that machine resources will never be allocated to stages where that data is not necessary so why waste a bunch of cpu cycles on a source doing something when when it's you know the flow won't be able to handle it so you know real allocation of that work and it makes for a better a better stream so that's akka streams and then things get especially interesting with aca when you bring aka clustering and cluster sharding and i'm not going to go in a ton of detail with that but uh you know the ability to have you know potentially different streams running on different nodes the ability to have information cached in memory you know stateful data cached in memory across any of these and start building um you know stateful streaming applications as opposed to simply uh state lists and then last but not least we've got apache kafka so kafka is the you know we call it a data back plane kafka is the pipe and in cloudflow we use it as a communication protocol so kafka makes sure that information is stored to disk and it also has really nice guarantees about you know splitting up this data you know so we've got a kafka topic each topic is broken up into partitions and uh data is um guaranteed ordered within a specific partition so uh overview of those things and then we can get into the the interesting part of this the new part which is cloudflow so apache spark awesome for windowing and aggregation does a lot of really great things with ml um but the problem is it's extremely high with respect to cpu and memory utilization even when you're doing the most simple things it's it's kind of a beast and has fairly high latency compared to a lot of these other things flink is better than spark when it comes to cpu and memory um cpu and memory utilization um it's got medium latency compared to aca and it's still great for windowing and aggregation akka much more low level streams really isolated inside of one node not a lot of windowing and aggregation extremely low latency extremely low cpu and memory footprint and this really nice reactive back pressured stream and then kafka is uh kafka is just kind of a pipe for for storing these these these messages so um let's so now that we have all these pieces and we've talked about you know all the tools that you could be using in um to build your streaming applications well the the you know cloud flow is really built to you know give you the ability to use the right streaming tool for the job less boilerplate code so less docker files less kubernetes ammo less um the spark code to read from kafka right to kafka or serialize or deserialize we've got more microservice best practices and significantly less worrying about you know the plumbing of our application much more focus on you know use the right streaming tool and write the business logic and uh you know just as much acceleration on those pieces as we can possibly deliver so that's where cloudflow you know it really was designed to meet those needs so breaking down cloudflow into um into how to actually use it and um what are the important pieces to be aware of when you start using it so cloudflow has two most important features it's got a an api for developing your application in your ide and so we've got apis for both scala and java at the moment and then it's a kubernetes operator for deploying this application into kubernetes so an import important part of cloudflow is there is a hard dependency on kubernetes cloudflow's designed for getting cloudflow applications i'm sorry yeah the kubernetes operator is designed to get these cloudflow applications running in kubernetes as quickly as possible so first let's talk about the api and and we'll go into some code in a minute and um and actually see these things working and do some deploying but um quickly the the main abstraction that you have to worry about in cloudflow is called a streamlit so streamlet is this um you know independently scalable stream microservice and the first thing you have to do to build a streamlit is decide what kind of engine you want to use so spark pocket and flink are the three streamlet types that we currently support um let's say we're going to build a we've decided on akka the next thing we have to do is define the you know how does this streamline communicate with the outside world what are the inlets what kind so you can have you know between zero and uh you know many inlets you can have you know between zero and many outlets so that's the data coming in the data coming out and the data has to have a specific type so here we've got you know we have data coming in as blue rectangle we have data coming in as um green triangle and then we you have to write the business logic that converts these data types into you know an output message of uh you know orange circle and everything that you do inside of this streamlet is 100 your standard akka streams api or or pure aka you have the complete world of akka available to you inside of this streamlet so and that's the same with apache spark and flink there's not a separate build of any of these things to use in cloudflow it's just the normal api cloudflow's just giving you abstractions on top of those apis to make make a lot of these problems easier so now that you've defined your streamlit you have to connect it to the rest of the world this is what we call blueprints so you define how data goes from you know we want our data to flow from this streamlet into this streamlet via you know these outlets and inlets connecting and cloudflow will actually guarantee strong typing on compile time for your applications that are being deployed into kubernetes and uh if you hadn't caught on already each one of these lines when deployed into kubernetes is actually going to be a kafka topic so each one is going to be its own independent kafka topic and cloudflow takes care of provisioning those kafka topics for you as well last thing i always um you know this this can be a little confusing but just because a streamlit doesn't have a defined inlet or outlet doesn't mean that you can't pull data from somewhere else you can um you know just like in normal akka you can subscribe to an external kafka topic or read from a jdbc database or or provide a rest endpoint the full world of akka is available to you um you know on your streamline so you can uh you can take advantage of that and um you know pull data from many sources one source and then you know bring it into this cloud flow streamline so let's actually look at something working so over here this is uh so this ui right here is our um uh it's a commercial tool from light been for monitoring cloudflow applications so i should be very clear you can deploy cloudflow applications into kubernetes um using the pure cloudflow open source but to use this monitoring ui um it is uh it requires a subscription um but it is a good interface to be able to see um you know these things working so we'll look at this one first uh real quick um so i have many you know three different cloudflow applications deployed and um uh from these applications we can drill into one of them and here you can see um our our cloudflow blueprint so the cloudflow blueprint you actually do you know when you you define on compile you know in your ide but we're able to actually see what's happening here um from a deployed application running on kubernetes so we've got a bunch of different spark streamlets um they're you know communicating with a kafka topic and then we've got akka streamlets and each one of these circles is you know each one of these streamlets is its own kubernetes deployment so it's an independent microservice that's running on its own and by using cloudflow to build this application and monitor it we're able to have a more of an eagle-eyed view of what what this complex streaming workload looks like and we're able to see you know information at the top level you know what's coming into this app what's leaving the application and we can identify bottlenecks and in issues or problems very quickly just by looking at these visualizations and then we're also able to monitor something called consumer lag and we don't currently have an auto scaler for consumer lag but hopefully it's something we build in in the future to be able to say you know for a specific streamlet how many records are you behind in processing and um how should we scale any of these in order to you know better work better deal with the total throughput so that's a you know a kind of a quick intro into one of our um cloudflow streamlets and i will actually if i can minimize this window somehow um we can look at some some code real quick so when you're developing a um this might be a little small so i will make the code bigger when you're developing a cloudflow streamlit like i said i talked about those those steps that you need to do i want to go to this one so this is a cloudflow streamlit built in scala and in order to you know the first thing you do is decide what type of streamlet you're building so i'm building an akka streamlet so i extend the the akka streamlit then as i mentioned inlets and outlets how are we talking to the outside world so we've got avro inlets avro outlets so we've got messages of um you know serialized avro of type consumer customer transaction model descriptor coming into this streamlit and then what's going out of this streamlet is actually a scor scored transaction so this app that we're looking at is is a fraud detection app that's taking in credit card transactions um and new models and uh and actually um doing ml on those models to be able to i'm sorry doing ml on those transactions to be able to decide whether or not it's potentially fraud and then once we've defined this this shape for our streamlet um we we have to define the akka stream's logic for how we how we go from those sources into that sync type and for those of you familiar with aka streams it's a pretty you know this is a um it's a this source with committable context is something that comes from cloudflow but it's just a standard cloud or it's a standard akka source and then we can write you know in this um oops in this flow right here you know it's just normal akka streams code and you can use you know in this whole thing i'm actually using aka clustering as well and distributed data um to be able to do some different things but um uh yeah that that um the flow code is down here so what i'm doing is i've got a model server actor i'm doing an ask to to get a score back from a from the model server so i send it a transaction message and then i get a response of whether or not it's fraud and then i move that forward in the application and we can actually see that in this blueprint right here this this streamlet was the one i was looking at right there so we've got customer transactions coming in new machine learning models and then what's going out is um potential you know fraud scores so that's a kind of a quick overview of um building a oh i'm sorry i should also mention um this is what one of our uh blueprints looks like so the one we were looking at a minute ago is uh the the fraud detection so we we defined these streamlets up here in our blueprint we've got our fraud detection streamlet and it points to this specific uh scala class and then down here we define kind of the interaction with uh you know how things communicate so for our transactions um i'm sorry here's fraud detection so for this fraud detection topic we've got the the fraud detection streamlet results are going are being produced into the topic and then the fraud report um you know ingress is uh is consuming from that kafka topic so this is really the blueprint is what you use to stitch all of these streamlets together and actually make you know this this higher level view of the of the application so um sorry i'm not i'm seeing everybody's messages up there sorry i'm not getting to them um i have a few more things i want to talk about not an enormous amount of time so um we'll we'll come to them all in the uh in the questions session afterwards so the last thing that i want to talk about that's that's super important is um is the cloudflow operator so you install this cloudflow operator in kubernetes and you give it access to do things like create namespaces create deployments and all the kind of management deployment things that you need for your cloudflow applications and this you know this way of deploying applications giving a an application um context to these operators is really really powerful so um you know when you're so after i built my entire app um the the stages that i would use to actually deploy it so i've got i can run it local i can run the entire streaming topology in um you know locally on my laptop which is which is really useful i can build the app including all the docker containers so i'm able to build this complex streaming application without writing a single docker file or a single line of kubernetes ammo and then i'm able to deploy using just cube ctl cloudflow deploy and then this little deployment manifest that's that's built that's created by the build file or the build um uh statement and uh and last uh if you wanted to scale things out manually um you could do it using just the cubectl cloudflow scale command so let's see this all get done real quick so i can do cube ctl um cloudflow i'll make this bigger uh maybe that's okay cubectl cloudflow list and that's going to give me a list of all my cloudflow applications that are currently running in kubernetes i'm going to do undeploy and i'm going to undeploy that fraud detection application and so now it's it's undeployed if we go back to our ui we should be able to see that it's it's spinning down and it should get uh get cleared out in a second yep there it goes so then if i was you know this is a clean build and i just want to go deploy it into production the first thing that i want to do is i want to test it locally so i'll do sbt run local and this is going to it's going to compile the application um uh compile all of my avro schemas into uh into scala classes so that i can use them it's going to build all my jars and it's also going to read my um my blueprint that i defined and make sure that the it's strongly typed and correct so that there was a statement right there that said blueprint verified and so it checked to make sure that the types of data that were coming out of specific streamlets lined up with the uh the front end of those streamlets so um this is that uh well this is the run local topology that gets generated so we've got our streamlets we've got our kafka topics and um you know all the way through the entire application and then i can come down here and and see so all of the information so all of those streamlets and all the kafka topics and spark and everything are running um on my laptop right now so i can go into this log and it's going to be an absolute mess because there's just a lot going on right there so i can go into this file and i can do tail um there's the ocalogs and there should be a yeah there's a spark log in there too i don't know if that one actually prints anything though yeah a little bit of spark stuff so that's me running it local and now i'm ready to deploy into production so i want to i just clicked enter to end this run logo and so i want to do spt build app and so sbt build app is going to do a lot of the same things it's going to build the jars check the blueprint but then it's also going to build a docker container for each one of my streamlets and it's going to push that docker container to the repo that i've kind of designated um in my application so right down here i've got my target environment dot sbt so this is it's pushing to this docker repo and then i'm when i'm ready to to deploy it'll actually give me the um uh it'll actually give me the um the command that i need to deploy this into kubernetes so luckily all these these docker pieces should be should be pretty quick because i've already put all these images up in uh up in this repo but uh yep this well just for a point of time i'm gonna just go back to where i deployed it a minute ago and it appears to be back pretty far there we go um so that's the the deploy statement so there's just a um yeah i use this uh target file that's json that's just kind of a manifest and then um now that that is deployed we should be able to go to cubectl cloudflow list and see that that fraud detection is is now starting up i can can look at the the status of all the the containers that are starting up so still pending should start up in a minute and be ready to go so cool i uh timed that pretty much perfectly despite a little bit of rushing but uh um a lot of a lot of info to go through and i've seen i've seen a lot of questions in the in the chat and i'll be happy to um yeah i'll jump into the um the questions uh session and and be able to kind of talk through um any of those that i didn't didn't talk [Music] through you