Scale By The Bay 2019: Ryan Knight, 8 Keys for Successful Serverless Architectures

Scale By The Bay 2019: Ryan Knight, 8 Keys for Successful Serverless Architectures

Recording: Scale By The Bay 2019: Ryan Knight, 8 Keys for Successful Serverless Architectures

[Music] so this talk was originally best practices of Sur elicited this week I've had a chance to talk to a number of people who kind of have helped me rethink of what service really is in the future surrealist and so last night I rewrote my entire talk so this is I wrote this about midnight last night and completely redid it from service best practices to be the future of service and the reason I did that is because I feel like what we talked about today of service is a very limited viewer serverless you know the panel last night there was you know it's a pretty scathing remarks and surplice quite some colored commentary that was pretty interesting so really if we look at what services today it's really about fully managed services right from you know like application what types of databases to security these runtimes like completely encapsulate everything we need to do something like run you know like a Java function or there's Google bigquery that has everything you need to query a database you know process the results so this is all about you know authority the management of these server tasks to the cloud provider and the focus is really on the ease of operation development today so the current value service tonight and I personally you know if I was on the panel I would wait and you know there's some value in service today I think beyond just the ease of running and playing application there's a number of areas where it's really useful what is security with the least privileged approach right if I have a cloud function and I only give it access to a single database if that cloud function gets you know attacked I'm going to minimize the blast radius my service services so I think there's a lot of value to serve lists in the security space Netflix uses a they call it the trough I don't know if I spelled trough right but right what they have is this ability to scale down their services and make the resources available for things like write machine learning so at night or off-peak hours they can take the excess resources and do such things such as video codecs processing so this ability to kind of balance my workloads across the service is very useful I also think there's a lot of valued service for the gravia there's talks yesterday Jason David great talk yesterday about the cropping up you know how to optimize his cold starts and image sizes and Prius runtime performance so what should the service be get us where we're at to be today we're really focused on the skis and development and operation I think the one of the few really the focus is going forward is going to be not only on the C's of development operation but also on the economic models around server lists so it goes beyond the season operation to how we can optimize their billing models so really this this word service is becoming so overloaded maybe somebody thinks I think we're gonna at some point someone come up with a better name for serverless but really it's about this more than abstracting over servers it's about this you know James Ward said last night you know a model that matches supply with demand so depending on how much compute I need that demand is automatically meant for me and I don't think about how those resources are provided or how they get allocated the other way of thinking about this there's an it there's a really interesting paper on this the perfect years service computing or it might be couple computation storage which is already - most cloud providers already do this but not only doing the couple the computation it's towards I also provision a price that independently so my stewards gets priced independent front with how I compute it's priced so there's this opens up some really interesting possibilities for new avenues of you know billing and you know economic unit resources you know the economic models behind computing and really when it comes down to is and I think the key of service in the future is going to be why I want to pay a proportion to the resources used instead of the resources allocated so you know today when I spent up a crew and a DS cluster I saved the known sizes and how much compute I want whenever I'd have to specify the amount of I want ahead of time I'm having to pay for those resources have notified and so you know today there are you know like land as those kind of things you know bill per gun you know like the 100 millisecond marks but is that Billy models expands across the stack I think that's where it's service is going to go and we know the bill for my resources allocated across the entire stack you know in terms of storage there's all the different layers there's a fundamental Achilles heel in a service a though and that is there's this major I think you know hurdle to having service being adopted today and the there's one of these challenges is it's almost impossible to auto scale your data store so if you're familiar with Cassandra or like the Cassandra's I mean I love cuz hat it's amazing database but it doesn't I can't auto scale like Cassandra I can't you know we had five nodes you know for an hour in the sky back down to three notes it's impossible to auto scale like he's having you know ElastiCache Redis I have to pre-allocate pretty much the number of nodes I want to make compute resource so this makes it really challenging to meet these flexible or building models where I can scale to zero when I'm not using something so you know these pricey models don't truly line up with the models of these back-end services yet today another thing if we look at the primary hurdles of service is if we look at the use cases where it's being you know where it's really useful parallel process of test low traffic applications stateless web applications right job scheduling these kinds of use cases this is where I think the sweet spot of services today but if you look at all these they have a common thread and that's their primary of the stateless so this is where surplice is today is on these typists stateless applications which leads a major gap because i've limited to how I can use my functions my service functions so what has happened is service has become anonymous or stateless functions so you last night's panel you know just made the comment that you know it still we have you know extremely low latency access to you know storage and from a server less function we're never going to be able to achieve state and this is the primary problem that we're facing today is serverless is acquainted with stateful functions yeah go ahead synonymous thank you synonymous with stateless functions thank you it's nice to have an autocrat to the audience I appreciate that so we look at the surplice today some of the common limitations are functions are primarily stateless the world short lived the problem is with this ephemeral short-lived function it gets very expensive to lose my computational context and rehydrate the process of video codecs they lose that process state you know machine learning models you know if I'm trained in machine learning model I lose my Stevie halfway through the training I've lost a lot of the processing so that context gets lost another really important problem is the durable state is always somewhere else so with the AWS land as a tip slab bespeaks dynamo let's see like it's very great because they do a lot of tricks to making a dyno really you know dynamo if I want to use any other kind of state connection pool but I'm having to deal with database connection well that state is always outside of that function so there's no way to really directly you know direct accessibility at my state because I'm always external storage so this really limited the options for how I kind of manage and coordinate the state and this also limits the options for Molony that my consistency guarantees so the current view a state of how that state gets managed is really limiting not only the usefulness of a study for functions but it's so see less functions also limited my Billy models so there's there's these really common themes that are living in how service is being used and adopted today so you know it really summarizes the primary limitation is no state so how many people write applications that never beat any state yes one person so there's two people two stragglers say that so the use cases where we don't need state are very limited to you know fairly limited use cases so I think for the adoption of serverless to become you know widespread we need to figure out how we get state into our service so what we need to do is we need to move from this model is stateless server list to state full service so the general idea of what people are going trying to do is how can be abstract over our state so if you think about abstracting every state this is it's a little bit of a stretch but you can think about it spark like spark collections abstract over that state and I don't think about the fact that that state is actually across a cluster I'm not thinking about where that state is actually being managed and stored I'm more just operating on that state and spark this match in that state for me underneath the covers so I'm abstracting over that state we have strike over a lot of things in compute today right we abstract over resources such as north kubernetes I'm abstracting over there you know compute and memory sto service messages abstract over our communication layer so there's this is a you know fundamental paradigm of computing is how we abstract over these different layers to make it easier to use and leveraged inside of our applications and I think the future of service is going to be the same notion of how do we abstract over our state so that we can make it easily accessible inside our stateful serverless functions and so this these towards having our statements by a framework for us and not you know us directly trying to manage that state you know in traditional crud operations so the reason you traditional crud debased models won't work is there's a number of reasons why I can't use like a condition like an Oracle or you know even like a Cassandra is because it in a service world it's very large catchment database connection pools database yes a connection pool and it's a short of operation immediately shuts down that's that's a lot of overhead for a very short that function also if I have you know like you know in scaly know several hundred you know service functions and you know the second time period and they all make connection to the database I'm going to cause you know and run out of connections the other problem is that if I had this unconstrained access to my database if I can do you know you kind of crud operation there's no way to automate these operations what we like to go towards is in a way of how can we automate access to the state so that the programmer doesn't have to think about it so if we can limit the act database access patterns we can start to think about ways of abstracting over that state and making it easier to work with so this Lee so what this means is if I can understand the intention to be taxes so if I had a way for my service functions to say hey this is the type of state and these are the access patterns I need you go advantageous for me I can then start to figure out ways to optimizing this deena access patterns so the the frameworks could provide ways you know automatic the cache might need you know you know change my consistency constraints for me you know pre-caching the data that I'm going to need you know dealing with resilience even easier for me if I had a way of decline the state those needed and how is going to access it and work with it so to get to this model of abstractly overstate what we really need is better models for distributed state you know most people the scala community are fairly familiar with you know events or scene event sources this model of append-only logging I get a you know I get a series of events at twice my application I walk those events so the traditional example is you know banking I did it you know with baking I instead of keeping a running balance in a database table you know this is my account balance I'll have a log of you know deposit $10 - $10 - $5 per throw $3 I have I cannot go and replay those deposits and withdrawals Adam hop and I get to what is my final state so I'm not maintaining it the actual balance I just maintain the event logs that have happened over time this the nice thing about this that bottle is that I don't have to have a like a strongly consistent database that's maintaining that balance for me now the other model that I can use is Co DTS who hears her to see our duties all right so I'll talk briefly about saryniti C or D T's are a really unique data model tonight I didn't even put in with Siri t set for a convergent replicated stream coming I'll try to link Thank You conflict-free replicated data types all right okay hi so sir duties the notion of a seer DT is data types of guarantee conferred to the same guy inspire network delays partitions and message reordering so the values that converge to a single value regardless of the order of the operations how things are coming in how the data is flowing through the system I'm always going to be achieve the same you know like account balance rate so if I have you know three deposits for withdrawals I'll always end up the exact same account balance no matter how those deposits and withdrawals are applied so C or D T's providing a distinct view of data right then there is much different than just like a adopt data store this isn't a place okay just to dump my DD and like official database instead this is more of an abstraction of my data tightening so I'm thinking about my date in a very different view so that what this is it's more of a data structure that tell us how to build the value and so because it has intelligence and how these values get built I can come up with new models are working with these values so some examples of convergent series convergent operations so associative right it doesn't matter the order the operations apply commutative sorry associative is grouping doesn't matter like how I grouped the operations together a commutative order of application doesn't matter I had a potent duplication doesn't matter so there's a lot of data types that match this so you can imagine like registers counters there's monotonically increasing counters like scores of a baseball game right these signs of you know you know like time series data gathering with weather data these simple operations I just want to you know Kendrick conclusive value but it doesn't matter how the operations are applied so there's a lot of value of C or D T's in distributed systems one is that they can replicate cake data across the network without any synchronization mechanism so if I'm using them in a surrealist function it doesn't matter how these functions are replicating the data between the different functions because they're going to converge the same value and the real value is that I can avoid distributed locks two-phase commit this type of thing so what I get is consistency without consensus so judicial databases if I were to try to you know use a traditional database with a you know a two-phase locking on a distributed system of you know several hundred functions I'm going to end up with a lot of you know problems in deadlock so the other primary paradigm that's being adopted is event sourcing and stateful service so event sourcing is an ideal model for service computing the idea right I talked about this right the event lock tracks which events have been applied and what is the current state so you can imagine that I can write my applications using event logging so I have a series of functions that provide you know that a a series of community functions or associative functions it doesn't matter exactly the order that these functions are being computed but when I run these functions they add up to the cumulative state that I need to get to and what I can do is is I execute each of these functions I can put it into an event log to say you know function a has computed this is the output function B is computed this is the output and at the end of the event log of my functions running I then get to a final state and if at any point my computation gets disrupted you know the I get an outage a note goes out I can see it from a snapshot point of view of what events have been calculated wherever I processed and where do I need to restart and I can replay that's pretty and so this model allows me in a stateful application to come up with a new paradigm for how I do the the programming in that stateless application this is Pat Helen he talks about he has a he coined the term asset 2.0 there's a lot of interesting papers out there he's rich and other people have written talk about this ashes asset 2.0 right so instead of the traditional asset model where I'm concerned with consistency I'm now concerned with figuring out how my operations can be associative commutative idempotent and distributed so now this the copy I depart is of all this is the fact that it depends on the shape of the data right in the level of consistency that's needed so not all of my data can be fitted into CRT keys not everything can be you know a soceity of data I should have put the accredit so this is this is from Kyle Kingsbury he has a blog he writes all about you know on Jefferson I know about all the different levels of consistency right so depending on the level of consistency that I need I need you know like strict serialize ability or die you just need linearize ability so if I need these higher levels of consistency these types of patterns won't apply so it depends a lot on the business use case and on you know how I need it and kind of sheet the deity's forms right so this isn't going to solve that every single problem and I don't envision like all of service will you know be able to you know everything will be on the moon to the service model but I think there's a lot of use cases that can move towards this new model of computing so if we look at how these so there's never enough like oh this is just a minute if we look across the landscape there's a number of companies and projects that are moving towards how do we do stateful serverless architectures and it's interesting to see that they're all converging on a couple of different primary paradigms one is the active base models becoming you know seems to be having like a resurgence it's becoming fairly popular cut across number least state full-service functions I'm also the workflow paradigm obviously is a very useful way of thinking about things right you can imagine a workflow where I have steps going through Z at each point of that workflow I can checkpoint my state and have that say - I'm at the checkpoint for me and if I have to restart the workflow you know it's somehow like an automatically ejected that oh well you're a sub now this is the state you're going to need so I really efficient that the future of staple is serverless and I think this is where computing is going is stateful surplus and I think this is going to be this major paradigm shift over the next you know a couple of years of this transition into stateful serverless so two examples of this something I just learned about this week from Colin is as your durable functions right they're executed using the azure function runtime they use what's called a a durable task framework and they maintain their execution state via events or C so they use this exact model of which functions have run and as I run that function store that function the computation of let's run into a durable state like an ash or table and then I can just replay where I need to picking off of that state and then they also and I don't quite understand this but are you going to use an actor like programming model underneath the covers so they do a lot of this similar like Microsoft has the or leads framework so I think that they're taking some of these paradigms of actor programming and applying it to durable functions so this is one example of it the other one is light bent Cloud State so you know lie pet for a long time you know how actors have been around for a really long time very popular you know programming model with actor persistence and clustering what's interesting is these building blocks of our key actors with actors persistence and clustering are being used to build a stateful serverless framework you really makes a lot of sense because they're this ideal framework for building saiful circus functions because actor persistence on top actor persistence there's aa distributed data which is essentially CR DTS underneath the covers so you can imagine taking these type of paradigms and applying them to the cloud and cloud state the general idea with a light cloud state is they have two primary models one is eventsource and the other C or D T's you can imagine I have a user function a user function it receives the command end and traditional event source of Technology I have a command that comes in an event that gets persistent so I have the command that comes in you can imagine a framework that when that command comes in I also know what date is associated with that command so the user doesn't think about that state what they're thinking about is what data needs to flow into that function to do this command and so that state is automatically ejected into my function and it's the output like basically midnight the reply with the events underneath the covers that are be persisted for me so I'm doing the computation you know based on these commands and the state is automatically being persisted for me underneath the covers using this event logging model the other model that white pick cloud state I do is doing is C or D keys and again C or D T's fit quite nicely with this model that I receive a message in or you see that message in I know a deed I want what's sitting you work with and so what the framework can do underneath the covers is it can automatically inject the deltas say that set state into my function that's going to be needed for computation the computation and my app that could be simply the deltas of that computation so I'm not having to do a complete crud operation but any meat I'm simply taking the deltas or the changes in and I'm outputting the deltas out and because I'm only doing the deltas this allows me to abstract over my state and automatically have it persisted for me by the underlying framework so the goals with the life at Cloud State is and they're driving towards a like a common specification for cloud state which I find really is very interesting what they're driving towards is having a general programming model for cloud city where you manage other ideas I have like in-memory durable session state across individual requests you know is a very low latency dynamic memory models so how to do a connection pool calling out to my deal with basically have my state almost like automatically appear for me as needed so this really reduces the latency and the overhead of dealing with state yeah you know some really interesting use cases around like collaborative workspaces leader election County so kind of the what perpetuated this talk was I saw this tweet from Colin and then retreated by Conrad and the same time I saw this three is constant so you know kind of in the same week I heard about Nash your functions I've heard about Cloud State and it you know I've realized that this is really the future of computing you know Conrad was one of the original Hawkeye you know whether regional programmers on the actors actor team he he talks about how stateful service is the next big thing right actors are bound to take over the serve landscape full stop and the reason is is because they provide that ideal paradigm for steady full-service functions so Jonas Bruun era his quote is the promise of stateful server list is revolutionary and grow to dominate the future of cloud so you know what we're seeing is a lot of thought leaders are saying you know how the future's shaping up is serverless is going to get a foothold and it's going to transform how we do computing we just need to figure out how we get state there I'll post the slides online there's a number of links to talk about the details so there's other flank history project there's Microsoft a per I don't quite understand a pert so there's a lot of different projects that are converging on this this directions thank you any questions gray cloth Thank You Ryan for a great talk sorry you got the spot Colin I thought it was quite brilliant but you'd vision kind of a new programming model stateful functions so I mean personally I think the approach that hazard durable functions is taking will will be the approach that most people will take it'll look like distributed stateful actor/model programming on this like substrate so you'll just model entities like there on the heap and you won't worry about you know where they're running and the maintain state you'll be able to stream data through them you'll be able to scale to zero I think that's an especially powerful model for especially in IOT modeling entities that way is just super powerful so it's it's probably going to be modeling stateful entities it's similar to an active paradigm in the cloud yeah one of those are people or transactions or orders or IOT devices yeah pretty powerful model great thank you so much calling thank you everyone for coming thank you everyone enjoy your lunch still after lunch so thank you [Music]