Scale By The Bay 2020: Panel: Were Microservices a Huge Mistake?

Scale By The Bay 2020: Panel: Were Microservices a Huge Mistake?

Recording: Scale By The Bay 2020: Panel: Were Microservices a Huge Mistake?

[Music] uh where microservice is a huge mistake uh and uh before we get to that uh i want to remind you that uh we vote before and after the panel so i'm gonna uh put the link uh about um the topic of the panel uh and this is the before link so basically before you hear any of us right if you agree or disagree with the statement uh please vote and after everything we'll take a second vote and so the team who will swing the most voters will win so please uh feel free to start voting in the link i put in the discord sbtb panel channel and also there is a optional field for the question so feel free to put some questions for the panelists in that field and we will use them so we have two moderators uh myself and willie morgan uh the ceo of co-founder of buoyant uh service mesh uh company uh the father of linker d uh and uh and we have four steam panelists so now we will introduce all the panelists and they will share their relations to microservices and after that we will uh put forward the motion and debate it so uh william take it away as an introduction oh i have to start okay well hi everyone i'm william morgan as lexi mentioned i am the ceo of a company called buoyant i'm also one of the at least nominally one of the creators of linker d which is a service mesh open source service mesh which is a piece of technology that only works with micro services so i am extremely biased and ready to cast legal challenges to any votes that i don't like um how did i what is my relationship to microservices i used to be pure and uh you know unsullied by these topics and then i did work at twitter for a while pete i'll see you there now um during twitter's kind of famous explosion from monolith to microservices although i don't think we had that word at the time we called it an soa and that was my introduction to this and everything in my life has basically been kind of following on from the service mesh onwards has been following on from that horrific slash amazing experience let's go in some random order of the zoom so nick is next [Music] hi um my name is nick schrock i'm the ceo and founder of a company called elemental and we're the people behind dagster which is that up there but in my previous life where i formed my opinion on microservices uh was when i was at facebook from 2009 2017 and effectively the early part of my career was spent as kind of part of the core team and one of the architects who our our mandate was to save our monolith and we succeeded and then one of the outputs of that was an open source technology that i co-created and co-wrote the spec called graphql which ironically is now an active participant in the microservices ecosystem which is uh ironic um i i look forward to uh contextualizing and more precisely uh defining uh the polemical tweet that spurned this discussion uh on to the next one thank you nick chris you're next yeah hi i'm i'm chris richardson um how did i really get into this micro service thing well way back like 2008 nine i created this um early pass called cloud foundry um and that was for deploying java applications on ec2 and that was actually a monolith um and then ultimately that got acquired which is and that's where the name not the technology of the the current cloud foundry comes from and then around 2010 i i read this book um the art of scalability and they had this three-dimensional model of scale of scaling and one of the dimensions the y-axis was functional decomposition which was like breaking things apart and breaking an otherwise large application into smaller pieces primarily to tackle complexity but also to deal with um load and it was like wow yeah you could do that and i looked back at the original cloud foundry and it was like that would have helped a lot um so i kind of just got interested in this style of architecture even though there wasn't really a name and actually was investigating it and in my role as doing developer relations at vmware um i gave a talk on this thing first time 2012. and ever since then that's all i've been doing um so i guess there is you know there is some emotional investment in there but i'm prepared to get over it eventually um so yeah i know so today i mean i normally would say i travel around the world working with clients which of course has since march has not been possible so i've been helping clients use the microservice architecture effectively from my off home office in oakland um my sort of if i was to say in a one sentence what my position is here it's like it depends right thank you very much chris uh kiki's next there so i am kiki carter i am a software engineer at live band and um my how i was swept into this world of microservices is it kind of goes back to my background um in consulting and in the consulting business you know a lot of times you get people who have read and and i feel like this should somewhat be banned but you get some executive who reads cio magazine and they say oh we absolutely have to do this and at that time the absolutely what we have to do was the digital revolution everything needed to be going digital we need to um do this digital wave and we're digitizing this and digitizing that and what that ended up really meaning was people wanted to modernize applications and they wanted to modernize them using sometimes using emerging technology and and you get swept up in this world of all of these words and things that have meaning and sometimes they they have not so clear meaning i put it that way and um one of those things we got into was microservices you know at that time it was soa um and those tenets still remain strong and and i i believe that those characteristics of soa remain strong how we implemented it way back when was not great and we've made another evolution and implementing it using this thing we call microservices and i believe we will make it yet another evolution in this world of microservices and may perhaps be called something else but i do that sort of been my journey and my arc into the microservices world has been around digitizing around modernizing and around really um evolving applications and systems and mostly not greenfield so places where monoliths existed and not only were there monoliths existing there were other legacy systems that existed and mainframes that existed and people who you had to play very nicely with so i did not come from a very clean and pristine rebuilding greenfield microservices world and and therefore i have a a beautiful view of it that's not how i i created my view but that being said i will put my um my stake in the ground here that microservices were indeed not a mistake and i will elaborate on why i feel that way and as you can see i'm from atlanta so i'm hoping to be able to turn this state a little bit you know based on based on this locality so that is me thank you very much kiki we'll see how how the swing world goes and pete is the last but not least take it away pete hey everybody uh i'm pete uh currently um uh a tech lead at twitter um but that's a fairly recent development for me for the past couple of years i was actually managing a pretty big org um within twitter as an em which included uh what we call health infrastructure which is like the rules engine that computes features and you know classifies tweets as spammy or abusive as well as all the internal tools that support health so you know reviewing content uh that kind of thing um a lot of uh uh within that that group there's like a whole spectrum of like monolithic applications and then highly highly micro service sized applications um so seen a lot there at twitter i came in via acquisition um my company uh smite was bought we were doing similar stuff uh trust and safety things that was we had a monolith we had a bunch of services as well um it's kind of a trans kind of hybrid architecture um and before that i was an engineer at facebook uh which is you know how i know nick and um actually worked on two monoliths there uh dub dub dub is um is a monolithic architecture as well as instagram uh so um you know cena i've seen a couple different iterations of these types of architectures the reason why i kind of started caring a lot about the uh this space was because over the past couple of years i've interviewed lots of engineers um and on like every resume they say oh you know we decompose the monolith into microservices and it this is from engineers coming from companies of like four people all the way to companies of you know tens of thousands of people and i'm like i'm thinking to myself you can't possibly all have the same needs and concerns and and you during the interview you start to ask questions and i started to get the sense that maybe we were uh overpivoting a little bit on this whole microservices thing um and that started to get reinforced with with some of the things that i saw um you know within twitter uh which which again has monolithic architectures that work well microservices architectures that work well and also monolithic architectures that don't work well in microsoft's architecture that don't work well so really excited to dig into this conversation um very happy to be here thanks for having me thank you very much pete uh so now uh we will move to the debate uh phase and it will work as follows i will put forward the motion which is recorded in this panel uh and and i will actually announce the vote which we have right now you will be surprised that voters are not this record i think in our democracy for scale by the way we've never seen anything like this we've had you know 73 uh responses uh so the motion is where microservice is a huge mistake and we will they spawn a multi-billion dollar cleanup industry uh as uh proposed by nick and currently so so our the team arguing for the motion so against microservices nick and pete the team defending microservices with team microservers are fine and peachy is chris and kiki uh so the current so nick and peter must tell you you have an uphill battle the incumbents are entrenched uh basically 75 percent are saying uh no they were not a mistake right so you have essentially your uh work card out in front of you and so now uh we will make the opening statements uh the team uh uh mistake i will call them team mistaken fine for shirt team mistake will uh open uh start with nick will frame the motion and then uh i would say kyries will as the captain of the team fine will respond and then pete will uh open and then kiki will uh respond and so after that we have a general melee and general debate uh using questions you can kind of debate with each other and we will add some questions from ourselves william and myself and the audience and in the end we'll do closing statements so nick uh open it up okay uh thank you so it was my tweet originally so i take responsibility uh i think the most important thing when discussing this topic is to precisely define what we mean by microservices so for example both kiki and chris use the term soa service oriented architecture and microsoft no no no no sorry i didn't oh i'm sorry two of you i've maybe blamed it uh use the term s.o.a i said it use the terms so use the term soa and microservices interchangeably i think there are two very different things so my definition of microservices comes from what i consider like the canonical article on this which is martin fowler's article which is services built around business capabilities and independently deployable by fully automated deployment machinery now the critical term in that is business capability do you decide to model your system across business capabilities and then put process boundaries between those business capabilities that is the problem i'm not arguing against services for example at facebook which i consider a monolith because as an application monolith facebook has hundreds of services so it will require service message and those those industries that i referred to will exist healthily and happily if lots of architectures like that existed i i am specifically talking about microservices with dividing by business capabilities because in reality the vast majority of the time what it actually is is a distributed monolith where you think you are separating concerns about process boundaries but you are actually inflicting enormous damage of your on yourself you make change dramatically more difficult you encode your organizational boundaries in process boundaries and in fact it's worse than that because organizations change all the time you reorganize you decide new product directions and so in fact the process boundaries in an org like take like uber for example which i believe has over 5 000 services which is for their scale organization in my opinion total complete madness and it's because that it encodes the entire lineage of your organizational history um you know in some ways ironically with uber i do consider microservices sort of kind of like a ideological libertarianism cause gone map where like you show up to uber your first day this is like back in the day right and you have like your tote bag you got your ann ram novel in there and then it's like oh we're going to have like a market of services and everyone's going to have their own programming language and there's going to be this like mutually beneficial transaction in the system that forms and i just don't think that's how the world works um i actually so i guess why i really am passionate about this subject is that i see teams using this so microservices are a tool there are cases where a monolith is so far gone maybe the people who could save it are gone that as an engineering leader i would recommend it my problem is i think is use probably two orders of magnitude too often as a tool right microsoft is a it's a tool it's not a goal and so why i'm passionate about this is i see lots of small startups who are building business apps that are not particularly technical novel technically novel and they have some small number of engineers and they have more github repos and more services than engineers and it's totally crazy they should build a web server and like a javascript app and that's they should be iterating with their clients but you know they've they're cargo culting more mature engineering organizations they want to use kubernetes they want to be able to tell pete in their interview that they migrated their monolith to a micro services so i think there's a lot of cargo culting and performative behavior here that actually causes enormous damage you can have separation of concerns without process boundaries you can have organization without process boundaries we have folders we have functions these tools exist they can be used so that's that's how i feel about the subject thank you very much nick that was a very powerful statement and i think really uh we all should remember with folders and functions uh i think that's that's that's really good to to keep in mind uh and to uh answer uh will chris richardson will open yeah well sorry you could say that there's there's like for you have to pick an architecture and let's just i mean i'm over simplifying slightly but basically the fundamental choice is do you have a monolithic architecture or a microservice architecture right i would just view those as two alternative architectural styles each with their own respective set of benefits and drawbacks and using and always saying use just one use the monolithic architecture or just use the microservice architecture of both the equally flawed sort of technical positions and the the architecture that you should use depends on your application at the current point in time like i would certainly agree with nick it's like yeah if you're an early stage startup you should probably you should almost always use a monolith right and only once you grow and actually have the problems of complexity and possibly scalability that the microservice architecture addresses should you then consider migrating to to that architecture um i mean you know so my i want to just clarify like so my definition of microservice architecture right it's a set of services which are basically mini applications right that are independently deployable so um and they are and they have to be loosely coupled right like building a tightly coupled microservice architecturalism is a major mistake and then in terms of granularity to me a good starting point is one service per team because the primary goal i view it is is enabling teams to be autonomous right and you should be structuring your engineering organization is a loosely coupled network of autonomous teams that are able to develop and test and deploy primarily independently and so by application of conway's law you need a loosely coupled architecture and and yeah i know if you're if your application is small monolith is fine if it grows eventually i think you're going to get the need for a mic for a micro service architecture um maybe one the comment around organizing around business capabilities is really interesting um i mean i know on the one hand yeah organizational structures evolve but i but on the other hand the core business functions are often i could argue are very stable right like i don't know i mean like an example off the top of my head like all banks have had the capability of depositing a check right and it's been that way for i don't know a long time right um and you know but how you've deposited the check has has evolved right from physically taking it to a branch to um scanning it with your smartphone but deposit check as a capability has existed so i would argue that yeah your organizational structure can vary but your core the your core business capabilities tend to be pretty stable and i and to me that is actually a good organizing principle thank you chris stop there i could go on for a while but i won't thank you thank you uh now uh opening for the team mistake uh pete hunt i love the word team it mistake like i'm being judged well i'm sorry you're radical extremists you're you're far far monolith cult people clearly yes clearly so i'm glad we're we're talking about uh conway's law um because i think that that is is really at the core of a lot of this you know um both uh chris and nick have brought it up already um and you know i agree with with a lot of um a lot of what you said chris uh but i would i would kind of take issue with with maybe um one or two points uh the but i think it really boils down to like like how confident can you get that your org structure is going to be stable i i guess that's the the underlying theme here is that if you're if you're going to bake the organization structure into your software like how confident can you get that that organizational structure is going to be stable and i think that it really depends on which teams you're talking about so you know for example there's a team within twitter that maintains key value storage uh key value storage service we're probably gonna need key value storage forever um requirements for key value stores aren't gonna change that much um that's a great candidate for something to put into a separate service um however there's a bunch of different business requirements that change you know all the time and in fact we we're since i've been here you know we've reorged a couple of times and i know that some companies reorg more often than others but every time there's a reorg there's a discussion of like all right who's gonna own this service who's gonna be on call for it um if we're gonna turn this one team into two teams you know now we've got to go match the the engineers that have knowledge of this service and put them on to the right team so it presents some challenges uh when you're changing the organizational structure and i also do think that business requirements um change more often than you would think i think gdpr is like a great example of this there's this cross-cutting concern with gdpr now where you know you have to be a good steward of of your customers data and you've got to you know put specific retention requirements on it and you need to be able to tell them where their data is and delete it when they ask and um that's a concern that that cuts across all of these different services and you might take a look at a new concern like that and say hey maybe that structure that we came up with a couple of years ago doesn't make any sense anymore maybe we don't want to be passing these strings of customer data around all over the place maybe we want to move to passing opaque ids and so i think that you know when you start to the more services that you have and the more kind of loosely coupled interfaces you have to develop the more expensive it gets to to change those interfaces uh when one of these cross-cutting concerns uh kind of comes in uh so that's you know my point of view on that thank you pete and now the opening statement from kiki yeah thank you and i would like to start by saying i i i don't think that it's fair to say that um soa is interchangeable with micro services i was given that example as sort of the journey of how we've gone with you know we said this type of architecture now this type of architecture and certainly something else and i also like to say just because something has been done incorrectly um and maybe has been done inefficiently and effectively or we've seen it done wrong doesn't mean that it is in fact wrong i like to go back to this interesting quote which is you cannot judge a philosophy by its abuses you cannot judge a philosophy by its abuses and we've many of us have seen abuses in microservices and also in monoliths themselves and i i'm not here to judge the monolith itself or the microservice by the abuses of either one because you can find those on both sides of the fence so i would lay it out like that um and what is a monolith but a service that you know if if it's a monolith that is the right size that is just for a specific thing then it is essentially a microservice and what is a microservice but also a monolith it is a small application but it is you know also a monolith so you don't you know separating them with these words and not with the intention is is causing sort of a confusion around this debate and so i don't think that microservices were a mistake i do think perhaps um in naming but certainly not in the necessary characteristics and it's wrong to suggest that we should not have evolved to deliver these characteristics at scale and in um distributed environments where where that makes sense um and meaning where your environment is distributed so to me i look at it as and other people have said this you know this is these are autonomous cooperative services and that's a lot harder to say than microservices but um that's essentially what we're after and my argument is that it's not for reasons that are completely obvious a lot of times you hear people frame this microservice architecture as something that's going to solve all of your problems it's going to help you get your speed to market it's going to help your resume building and it's going to make it easier for you to code fast and deploy more features etcetera etcetera but those are not the things that i feel like are most important believe it or not there are things i feel are even more important in this architecture than being able to say we we release coal to production every seven seconds you know companies who say that type of thing and what i see is very important in this system is being resilient and having this resilience due to being centered on truth and so what do microservices have to do with truth okay i'm taking this way out here now um autonomous services that own their own data in context have the means to present more truthful statements and apis and here of course i'm invoking something like promise theory or system stability at scale and data alone cannot answer those questions without context and state without data backing it up you know it cannot easily survive environmental failures and whether you're having a monolith or a microservice you will experience environmental failures um yes they are you know network partitions and things like that become more of a more of a danger in a microservices environment but failure exists within each of those and you have to organize your system in such a way and i won't say things like lines of business or anything like that but you have to organize your system in such a way that it can provide answers that come from as close to the source of truth as possible and that's not simply just state in your database you know state again state without context data cannot answer questions without context and for that reason i think when we talk about microservices we have to look at how we're organizing state how we're organizing the answers we give to people and and whether you say it's in the monolith or microservice however you want to say that and how we cooperate with other people who are who need that information um how we can say that this is true you know hey i'm telling you this information well how do you know it's true well based on these events or these facts this is the information and i like to to say that we need to be fact based or event based i i think that facts are important um i think that being able to derive views from other information is also important and again that goes into being able to collect and organize and understand events and in such aggregates and so i can continue i'm gonna stop but um you know we're all living in 2020 so it shouldn't be difficult to convince you that a system where truth is elusive can be easily destabilized and that is why i feel like you know microservices or that architecture are really basing it on on truth and facts and context around your data is very important and should not be thrown away all right i'll i'll ask a question now since uh we've gone through the opening statements and thank you that was a great set of opening statements i like how we have team truth on one side and team mistake on the other of course as we all know the truth doesn't matter the only thing that matters is getting the most votes um alexi has not described whether uh there's an electoral college system involved here or whether it's we're just taking the maximum number of votes the majority like you know uh some places do okay sorry you must stop with the elections it's too painful we're still living uh yeah interesting uh time to have a vote-based um uh system okay so uh you know i actually started looking at some of the questions here uh for the panel uh which were posed you know from from some people who filled out the form a lot of these are not questions so uh a lot of these are statements uh like huge is a strong word uh there's a question is this just click bait which i think is probably at least somewhat true um but actually you know i want to talk about the word huge for a second right the the title of this panel is microservices were a huge mistake and i think hopefully it's not too crazy to say that like most engineering decisions microsoft microservices are are a trade-off right like you're trading off something for something else so my question to uh the panelists is uh around the world huge is the trade-off that you make in microservices is that actually like an order of magnitude different from other types of engineering trade-offs say the trade-off between a nosql store and a sql store right is there something that's actually fundamentally different about that trade-off or is it you know just an ordinary engineering trade-off well i mean one one comment i would make is i i feel like the panel title is a mistake the premise that microservices are universally a mistake right like i don't know snorting uranium is a mistake right but taking aspirin is not a mistake unless you happen to take too much of it right it all it does very much depend man what parties are you going to yeah that uranium i don't even know why i came up with that one but you know there's sort of this implied absolute mistake or as constant mistake is is just really really not helpful i mean i mean one question to get back to the specific question i mean this is a major architectural decision right it's sort of top level architecture am i going to use a monolith which i mean that's so that's like a single deployable unit for my application or am i where am i going to use the microservice architecture a collection of services right and so that's a very high level decision and one of the traps if you use it inappropriately right is that you build a distributed monolith which is like all of the complexity of microservices with with all of the complex well with all of the friction of a monolith right so you know you go backwards right in terms of um actual productivity i mean interestingly that might get you a talk at qcon if you come up with a creative badly designed microservice architecture not naming names there but um you know there are severe negative consequences if you do it choose inappropriately and do it badly i think it's hard to know whether you're doing it well or doing it badly at the time that you're doing it though because a lot of times you don't know what the the changes and requirements are going to be you know a year from now or five years from now and depending on how many services you're creating and how different those services are i mean if you're creating a bunch of services that are all using the same libraries and same tech stack um and they're all deployed with the same tools and it's it's generally pretty unified um then it's kind of no big deal if you get those boundaries a little bit wrong right you can go and and work with that team and and move engineers around to go work on that other services on that other service but over time you know if we did get that division wrong and we have a lot of cross-cutting concerns between those those multiple services um we better hope those services don't evolve to to be really um really diverged and and then we start to see these like teams that are really specialized and have really strong ownership over that service um because that makes it hard for us to kind of like work across these services because each team has to to kind of coordinate at each step of the way i agree with that pete i i definitely agree that that makes it difficult to coordinate and it's even harder to difficult harder to coordinate in a distributed environment but i feel like those same difficulties exist with the monolith and if you get the boundaries wrong in a monolith or you have to make some major changes in a monolith have you seen the you know the insanity that ensues when you're when you have to tear the entire thing apart in order to bring the thing up you know it's like and and that's could be indicative of just bad code or spay spaghetti code or maybe you didn't componentize properly or organize your git repo the best way but those things happen and um if you do that in a monolithic environment you you tend to have to bring and and perhaps people are doing it much better but before i've seen you you tend to have to bring the whole house down to do a renovation rather than you know being able to renovate one room at a time and i feel like there is a trade-off there just to get back to that original question there there's a bit of a trade-off but it's also sort of a necessary um i feel like it's a necessary tooling or a necessary philosophy not calling things microservices or just throwing out tens of thousands and five thousand and hundred microservices but this idea that you need to create some really bounded context around something and it may not be business function it may be something totally different in order to deal with the scale of um just the scale of reality that we have here with today i mean we're talking about companies like we're talking about companies like twitter and this and that you said you've seen it with even smaller companies and when it comes to small companies to me i think it's even more important to have resilient services because you're hoping that you know you can get a funding round or something or you're hoping that you're not going to lose customers when you have only two or three customers you want that experience to be a great experience for them and so doing some heavy lifting on on the engineering side in order to give people a better experience on the user side i feel like is a trade-off but i feel like it is a necessary trade-off and it's one that has a better payoff in the end if if that's the case nick do you have a point of view uh no not at all uh just kidding uh the uh well first of all to chris's point i actually myself even though as the author of the tweet believed that it's not a healthy frame for a discussion like this so uh yes it was clickbait and you know i think i am guilty as charged as is alexi not to call someone out but um but you know i think what i mean by microservices are fundamentally catastrophically bad idea what i you know it more precision in a in a more productive framing i would say that the idea that microservices are the default and the norm and the ideal is a really bad idea uh and that it gets misapplied and is like kind of like the default thing to do whereas i think it should be the generally the exception to the rule in most circumstances like i like i said in the uh in the preface there are circumstances where if i was put in charge of an engineering organization and you had lost complete control out of the monolith that it's the right thing to do to start stripping it apart into process boundaries so you can actually horizontally scale your org or actually chris i thought your example of a bank was good like when some of the business capabilities are that stable and in fact you could think of them as even outsourcing them to different companies like that's appropriate and this happens on the inverse too when like you acquire you know one company acquires another thing and i would never advocate like regardless of context or situation that you immediately like rope that equa that acquiree into the acquirer's code base right so um so that's one thing i think actually you know one word that i heard a couple times was autonomy and uh jake donnam an ex-twitter engineer has a fantastic talk uh where he describes the trade-offs between these different approaches and he describes the trade-off as between leverage and autonomy and i think that is a very very fair way to frame the problem because from my perspective given the choice between leverage and autonomy i choose leverage every time because in my opinion leverage optimizes for outcomes and the value you can deliver to your users and stakeholders um whereas um you know in general people who advocate for market services architectures and very granular ones often really emphasize autonomy which i believe is a local optimization rather than a global optimization um it's a fantastic talk by the way i highly recommend it and but by the way i'll actually be a much less polemical framing by which to uh uh set up a debate like this yeah i mean i'll give you an example as a consulting client um a few years ago see the cio of the organization read an ebook that i wrote on microservices and then went back to his organization and said okay people we're doing micro services and this was an 8 000 person organization which was very hierarchical so the boss spoke it percolated down the hierarchy and then when i i was brought in to do some consulting i talked to these developers and it was like why are you doing microservices because my boss told me to and it actually it got to the point where um doing microservices was a kpi so their bonuses were like or basically i thought you know function of the number of services that had been written and that that that one story [Music] caused me a couple of years ago to give a talk on um anti-patterns of microservice adoption which is like doing microservices just for this well first off it's a magic pixie dust but then doing micro services just for the sake of it i mean it really you know as i said at the very beginning that your choice of architecture depends on the context at a given point in time blindly it seems like the this is like the perfect crime for you you you gave them this book they read it it was so good that they started doing microservices and then you had to come in to help them that was like oh cunning kind of yeah no i i just don't like that i mean they misread they missed because i mean like so the other thing that i am seriously into is is the concept of a pattern right because you know that's a recurring solution a reasonable solution to a recurring problem and the key part of that is you have the benefits and the drawbacks and so the which help you decide whether that's an appropriate solution for your context and if you skip over that part you you're just you're yeah you're on a path to trouble so i really like this talk of uh of trade-offs uh the next question i'd like to pose is um are there changes in technology that shift the decision surface area so for example i am fully immersed in the world of kubernetes and containers and cloud native and all that stuff and one of the arguments that uh that i've heard is that oh microservices are a lot easier to run because now you know by the time you've learned kubernetes which is certainly like non-trivial to learn running one service is kind of similar to running 10 it's kind of similar to running 10 100. so is that a is that a realistic argument or are there other types of technologies that might change the decision surface area between should i use microservices or not um i can i i think that's an incredibly insightful question um and it's really on to something because um i think a lot of the problem with monoliths and in today's world is the technologies they're built on top of so uh traditionally when people are building monoliths out of the gate they're using fairly heavyweight orms like django or rails and i think rails in particular is a huge problem for companies that want to organizationally scale a large software program uh it doesn't have to be yeah i've you know the twitter famously you know effectively had to blow up their monolith because they were unable to manage their rails monolith um and so i think i think that underlying technology changes if a framework emerged that was pure software that that laid out a vision of what these models could look like i think would go a long way towards addressing this and similarly like kubernetes makes it much easier and fun to spin up lots of services um and therefore certainly contributes to the popularity of microservices and actually if you look at the the google trends for from microservices it starts its hockey stick right um when kubernetes is released uh correlation is not causation but um i think that's something to do with it uh you know for for me um my kind of criticisms in microservice architecture actually have very little to do with um distributed systems concerns and like operational concerns because we do have great tools for running lots of services at scales we have kubernetes we have service architectures we have distributed tracing for me it's about the cost of cross-cutting changes at the end of the day and um and it's i think it's almost more of an organizational uh problem that microservice like if microservices are the default thing that you reach for by default you're baking your org structure at that moment into your code base more aggressively than if that's not your default tool and um for changes that fall like kind of cleanly into that structure and cleanly into that that delineation of services or your org those are generally going to be your cheaper changes to make right like they're that's kind of your median change is the way i think of it and i think a lot of where i kind of see the disconnect between myself and a lot of advocates for microservices is like i i see the case for the median change getting a lot faster than microsoft architecture what i don't see is the case um is the the tail latency the p99 change those the the bigger changes that your organization has to tackle are usually dominating the resource allocation that your organization is putting in you know like the changing the color of a button versus changing your data retention policy across the whole org like one of those is way more expensive than the other one and one of them cross cuts across all the services and so when when i advocate kind of against microservices again it's not like microservices are universally bad it's just that it should be a sad day when you have to spin up a new service not a happy day and um and the reason for that is because i think that every new service you bring up that p99 gets a little bit higher and just like we measure p99 latency as our kind of core metric for success when we're looking at a distributed system not the median latency it's because it's more indicative of like what is going to dominate the runtime of your of of your delivery um so well while i do think that like um schools like kubernetes mesos have really reduced the cost of the operational stuff that's not really my main concern about it uh certainly not in 2020. an interesting thought occurred to me the other day as a cup and i i want to get back to your p99 change thing but it's sort of like imagine you have a massive application right so there's sort of so one strategy say is to break it up into services each with their own deployment pipeline right so you in other words you're reducing the sort of the scalability required in your sort of in your deployment pipeline technology right whereas if you stick with them and but at the expense of sort of operational complexity and many many more moving parts in production whereas if you have a monolith your your operational site is kind of simple because you have one application and you just run it behind a load balancer but you then have all of this complexity around your deployment infrastructure so that you can parallelize your builds and have this con you know the build test and deploy of this massive monolith x um take a very short amount of time so it's sort of like there's there's like complexity any somewhere in this whole process and the other one is if you got a massive code base right i feel like cross-cutting concerns are really hard anyway right like you might have to you see you've got a gazillion modules you you kind of have you might have to implement data retention policies in each one regardless of whether those modules are packaged together as a monolith deployed independently as services right so i mean i i kind of just feel like if you if you've got a big system it's complex and complex stuff is just hard right no matter how you've sliced sliced it up except for the fact in the case of microservices my world as a service developer is considerably simpler because i can ignore the rest of the application apart from the immediate dependencies of my service but but i think the way that it ends up playing out in practice often is um yeah by the way i totally agree with you that like if you've got a big cross-cutting concern it's going to be painful in any architecture and i also um agree with you that you know you're in terms of deployment and operating a monolith versus micro services it's like you're trading one type of complexity for another type of complexity in many cases so you know operating model of sufficient size is hard too um but uh you know with respect to these cross-cutting concerns i think it's great as long as all the different microservice teams are like using the same abstractions and using the same technologies and using the same shared libraries and i feel like just kind of anecdotally when you've got a microservice architecture and these really strong bounded contexts surrounding them and teams that have really strong ownership over those services they tend to diverge and do things in different ways and that reduces the fungibility of engineers in the organization to go be able to like work across these different services and then when it comes time to implement these cross-cutting changes you know it becomes a really hard cross-team coordination effort you know you can't like just spin up a team to go work on a business problem anymore you got to go file a bunch of tickets on 20 different teams to go get anything yeah i would i see i lean towards just having a consensus-based technology stack that evolves slowly based on the changing sort of mark what's current in the marketplace as opposed to the individual whims of i want to use haskell today because it's cool or rust sort of decision making well i think that it's interesting that we're imagining that oh because of the technology if everyone's using the same technology then it makes it a little bit easier for us to do these cross-cutting concerns and i i don't know if that's necessarily the case that it becomes a little bit easier it may be because you can move people around possibly that could be a reason why it's easier but i think that um pointing to the tech stack and even the tools that we use to to operationalize and and whatnot is is kind of maybe there's a better way to think about it than that and i would posit that maybe we should be thinking about it in terms of you know having common protocols and i feel like that is something that is emerging and rising out you know we're looking at more common protocols ways that we can talk to each other um ways that we can um make progress without all using the same tech stack and i think that you know being able to like not being i don't want to say stuck with the technology but not being distracted by the next shiny tech for example haskell because you know haskell for your fun i i i think that that's also distracting for teams um and when it comes to the mess massive mess that we will have to clean up because on that i do agree um with nick's statement that there there's been a lot of damage route by not microservices but what i would call the abuses of that philosophy and i feel like you know we we still i still don't believe that that's something that we should give up i still go back to the point of if you have a well-formed uh monolith and it's doing one thing doing one thing very well then it is essentially and it is an alternate autonomous service you may use different words it may not cooperate with anyone but the user but it is an autonomous and cooperative service and if you had a single microservice it would just be that a monolith so i i don't think that when we talk about those cross-cutting concerns or changes that have to be made and what's the expense of a change it's it's not really around the things that you may think it's around and in reality cross-cutting concerns and changes that have to be made on massive applications typically come down to people organizations politics and the human beings involved not the technology because everything technological has an answer we have answers for all the things except for the people thank you kiki on this great note i think it's time for us to wrap up and uh i invite each of you to make one minute closing statement we should end with and this is why you should vote that microservices were a huge mistake or and this is why you should vote that microservices were not a huge mistake and so the link for voting is already in the chat if i did discord guys please start voting and we'll go in the same order actually we'll alternate the order so let's start with chris uh closing statement one minute oh okay yeah so you know so the world in general and the markets within which businesses operate is incredibly dynamic and unpredictable covert prime example of that businesses need to be nimble which in turn meets need requires it to deliver software rapidly frequently and reliably and sustainably over a long period of time because these applications can last for a while if you read the accelerate book that that will tell you that you need an architecture that's loosely coupled modular deployable and testable so if you have a small application small team a monolith is is usually an excellent choice but once your team and applica your application and its team grows right you in order to have this loosely coupled modular testable deployable architecture you off you should start look you should consider using the microservice architecture so i i would say that claiming microservices are a mistake is itself a mistake chris uh closing statement from the opposing team from nick sure um so the the claim is not that microservices aren't a bad idea all the time the claim is that they're in modern day practice a bad idea most of the time um and that i think to pete's point people who are making the decision to adopt them believe they're making a technical decision but they're actually making a profound organizational decision that'll be nearly impossible to undo um there are organizations where it's appropriate to do microservices or some sort of services whether they're micro or not um you know amazon like makes a lot of sense to have lots of services they made an explicit organizational choice to do that that has very explicit trade-offs most applications in the world are not amazon most applications are delivering a single application ontology and it's very important for it to be to deliver it quickly to make changes quickly to iterate fast and to deliver for your users so that's why i think it's a huge mistake because i think people are just kind of copying other organizations who operate under dramatically different constraints and using the tool inappropriately and i actually think it has profound consequences for those companies that adopt it uh closing statement from kiki yeah i think that if you look at microservices and perhaps if we just say services our autonomous i like to say cooperative services architecture if we look at this we can see um like i say philosophically there's nothing wrong with microservices i do feel like it's a a very strong statement to say that microservices in and of themselves are a mistake and then to back that statement by saying because people have done it wrong people have made a mess people haven't been able to wrangle these services themselves and and i agree that those mistakes do exist in that people have design services and organize them around organizations where they probably shouldn't have done that or business functions where they may not have made sense but that in and of itself does not make microservices as a philosophy the wrong thing to do um i feel like you know moving fast is not always the aim um this is not a world where we need to you know i know people say move faster and break things but it matters when you move fast and you're not truthful when you're the first to post a message or post some news and that news isn't actually true news right and the same can be for your services that you're creating to be you know we're moving first and we're moving fast and we've created this service and this service actually isn't doing a service for anyone that's not we're aiming at here what we're aiming at is being able to deliver systems that have high resilience that have great stability that are able to make promises that they can keep and that are able to interact with your customers your users other users of the services in a way that is um truthful in a way that is productive in a way that moves you that makes progress um within your company and ultimately helps you to be profitable and hopefully kind to the earth at the same time and so for that reason i feel like we should remember as you vote we cannot judge this philosophy by its abuses and for that reason you should make sure to vote that microservices are indeed not a huge mistake thank you very much kiki and now pete your final closing statement sure um i agree that uh you know we shouldn't judge a an architecture by its worst applications um but we have to keep in mind that people collectively will always make mistakes and we should set up architectures and systems for them to to fall into the pit of success uh rather than um you know tend towards the mistake i think that microservices architectures make cheap changes cheaper and i think they make expensive changes more expensive and i would much rather my expensive changes uh get cheaper even if it means my um my cheap changes have to be a little bit more expensive i think just because rails led to a bunch of really bad monoliths doesn't mean that the monolithic architecture can't work with the modern tools that we have today got stronger static analysis tools better development environments that kind of thing i think microservices are ideal in a world without reorgs and changing requirements i think the instant that you've got a requirement that requires changes to two or more services you've introduced a lot of pain and for that reason i think that services should be one of the last tools you reach for they should be a last resort rather than something that you immediately reach for doesn't mean all microservices are bad um doesn't mean service oriented architecture is bad it just means that they should be a last resort uh rather than the first tool that you reach for that's my statement thank you very much pete this was an amazing panel uh now before uh so we have now the results from the second voting um and before i read them out let me tell you that we have a q a uh uh here so the way q a works i'm going to paste a link and after i declare the voting results uh let's all click that link it will bring us to another zoom where the public can join us for the questions so so feel free to um to join the q a if the q a is in a separate room uh from the tracks so the tracks will resume in 10 minutes so have about 10 minutes for uh q and a's themselves and if folks want to hang out with the panelists you can still be in the q a room because there is no time limit in that room until the next talk is over so now let me tell you what uh the democracy uh decided so first of all i must say we've we lost about half the voters we did not lose the audience but uh it's a complicated subject so we have uh the after vote is about half the original vote so the original vote we had 98 votes and we had 77.6 percent that market services were not a mistake that so the team mistake essentially is trying to defeat you know disrupt the existing order in the original wall the team mistake had 22.4 percent and in the final vote we have 49 exactly 50 percent of the vote but we have team mistake now has 26.5 percent of the vote so uh based on the original criteria which team swings the most votes right we can say that the team mistake managed to wrangle about four percent increase in the group of people skeptical about microservices so they did make substantial inroads into the incumbent territory but on the other hand uh we collectively lost about 50 of the votes so uh it's interesting you know how do we judge this i think it will be a question of interpretation like we all lost american people lost we confused people people need to send in their mailing ballots they need time to think they need to send in their mailing ballots and then we'll know so that's that's good for me i'll see you in court uh with that let us all click the link which is at the end of the chat uh guys so it will bring us to another zoom it will ask you to leave and join [Music]