Nitro Tech Talk: Erran Berger and Tiho Bajic, Engineering Culture

Nitro Tech Talk: Erran Berger and Tiho Bajic, Engineering Culture

Recording: Nitro Tech Talk: Erran Berger and Tiho Bajic, Engineering Culture

we've been really really great hosts to Iran here today because we've got him in we got miked up uh we got like interviewed and shuffled between rooms and we've not thre him yet this sounds like my interview at linked by the way like 8 hours no lunch yeah and he's like just looking at all this food and he's like when do I get to eat and I'm like bro you got got pass C first okay if you can make it through a whole talk without you know having to had any L you know you know you're you know engineer you've been through through you know highs and L so uh let's keep this off um it's my great pleasure today to be introdu uh he's actually somebody that Greg house knows uh socially and somebody who Greg has Rec based on uh Iran's contributions you talk circles around engineering culture specific as it releas with cultural discipline and what it means to produce as a as a team scale so uh for the format of this uh Tech do SL session today we're going to uh quickly introduce iron and and his kind of uh uh accomplishments or basically the the sizes of the teams that you worked at and the products that you've seen um it's just set the stage for that so people can kind of hopefully better relate to some of the points you're making um and then we're going to go through each one of the six points that aon's been talking about publicly when it comes to the culture of of engineering cool uh first of all thank you guys very much for having me here today uh super excited uh to be here Greg like like too said Greg and I have known each other for a long time whenever we get together we actually talk about uh engineering and and uh our experiences at various companies I think between the two of us we've probably been on you know over the Arc of our careers uh at at companies large and small and we've seen a lot of different ways that that engineering teams get stuff done uh which was kind of the you know the impetus for me coming here today to share some of my experiences based on a conversation him and I had had so as far as my intro uh I lead uh our uh lead an engineering team at in developing our content products today our content products are focused on our things like pulse and slides share the publishing platform where professionals can share their insights with other professionals on the on LinkedIn uh sharing of of uh URLs and rich media content on LinkedIn as well as our our groups products where professionals connect with other professionals and and have a discussion about the challenges uh that they're facing and seek advice from other professionals who who might have insight to share with them uh prior to that and that team is is distributed across here in New York City and is about 150 Engineers prior to that uh I was working on what what we call our core products engineering team at LinkedIn products like the profile products like growth products like uh the entire member-to-member connection ecosystem things really what you think of when you think of linkedin.com really the the set of Suite of features and products that read and the rest of the founding team thought of when they when they created LinkedIn and I worked on that team for four years and I really saw that team grow from you know about 10 8 n 10 Engineers when I started in 2009 to again when I had the opportunity to lead it uh uh roughly 125 people and seeing and and we were really founded that team was really founded on on a lot of these principles we're going to talk about today we when you're running the core part of of a very large business on when I say running I mean I mean on the technology side it is really really important that uh that you take you have discipline and how you execute how you think about product strategy how you think about dealing with production issues uh and a few other and a bunch of other things and with that really Firm Foundation of these principles we're able to scale like I said really smoothly both you know in terms of membership from 30 million when I started to to north of 350 where we are today in terms of load you know from something on the order of maybe 1,000 QPS on our our uh services that were exposing our profile data to north of of a million QPS uh these things don't just happen by chance and I think being thoughtful when you're when you're a smaller team and having these as foundational principles really lets you scale pretty smoothly and now as a transitioned to this new role where we're talking about much more Venture products for LinkedIn less mature I've I've really taken what I learned during that that process and that growth that I went through and applied it to this team and I think we're we're starting to see dividends there as well well thank you very much for that um one one thing that kind of stood up for me as you were you were recapping that is is reflecting this morning's conversation we're having just in this room with the broad team about X right like what does it actually take to go 10x from membership perspective from the load system from the number of Engineers on the team right like it sounds like in all these Dimensions you you've seen that ten grow yeah that's a really good question um as it turns out you know what it takes to grow the team 10A or what it takes to to scale to grow your user based or customer based 10x or to scale your systems the actual tactics of doing all those things are so so different the I think the common threat is is taking taking the time to think about what it's going to take and putting together a strategy uh as best as you can as you can assemble right we have always dealing with incomplete data but as best as you can assemble based on the data that you have about what it's going to take figuring out what the first step is taking that first step you know measuring so so instrumenting that process again whether it's hiring or it's you know uh your your uh instrumenting your your systems and services whether it's instrumenting your growth funnels you know instrumenting along the way when you're taking that first step then evaluating and analyzing how you did against your expectations you know if you fell short or you exceeded what does that how does that inform uh the the the next step and and reflect upon what the original strategy was what was the original goal maybe it was the wrong goal maybe you learned something and now you need to change change direction uh one of my favorite quotes that that Jeff talks about um uh a lot is that you know a one degree I'm going to butcher this but basically effectively like when when you're trying to reach the moon like a one degree Miss in terms of trajectory means that you're going to be hundreds of millions of miles off when you when you when uh when you finish traveling and so it's really important that every step of the way you're reflecting again on how far you were or how close you were to the mark before you take your next step yeah so kind of agile I guess right yeah absolutely agile yeah absolutely taking stop of where you at recalibrating and then taking the next B step forward right so uh let's talk about those steps forward right and how you apply discipline to those right and and just going to switch quickly to the slide with covers uh basically the six talking points and this guys talk to these things it's meant to be kind of a Q&A I'll try to ask a couple questions all on as well but we have we have a spare mic here too so if somebody wants to ask a question feel free uh we have about 30 minutes ook for this so I'm thinking it's 5x six uh five minutes each um so um here in Nitro we say that our top uh engineering cultural value is technical Excellence right and and and and Jacko and a couple of other others have been talking about that that relationship of tech technical Excellence with respect to craftsmanship right because craft craft is like you got some signs or you got some course skills and you got practice that's kind of crap sure yeah and and you know let's just quickly cover um uh the value of craftsmanship uh with respect to how much should Val the execution well I would love to hear from you guys what do you think craftsmanship means in terms of your guys's day-to-day here I have my my opinion but thoughts anybody have a question to that okay okay is how do you reconcile the pursuit of high craftsmanship with moving fast and breaking things with the I not to skip down too fast on your slides but tremendous s of urgency with high value of craftmanship how do you reconcile thate repeat the question for the report yeah so well you have oh because I have the video got it how do you reconcile craft mhip with moving quickly uh and you fast failing and and also having a tremendous sense of urgency and everything that you do uh super good question I think that you start like we we should start with a principle of building things the right way and then you know making tradeoffs between that and speed of execution right so if we always if we don't we don't even ever think about well when we approach problems you know how do we uh solve this problem such that uh you know again given the data that we have it's it's built in the best way possible this user experience is designed in a really intuitive way um you know and think about you know the cost of that and and that doesn't need to be a multi-week process that could be you know gra jumping into a room and going to the Whiteboard and spending 30 minutes thinking about the problem if we don't go through that exercise I think that the the downside is that we wind up incurring a tremendous amount of uh of debt debt that's going to have to be repaid debt whether that's in you know uh unstable systems whether that's in you know uh a poor brand that your your uh product has developed with its customers because it's constantly breaking or the user experience is not inuitive uh and and so there there's actually a really high cost of not going through this process now you know applying once once you've done kind of that diligence and thinking about craftsmanship how to build this thing the right way whatever it is it could be user experience and building a really beautiful user experience or it could be you know thinking about how we scale this thing and do we actually rewrite the system or to just throw cash in front of it uh because we needed to scale 10x then we can have a discussion about well you know what is what are the business needs right and and trading off the business needs like against the timeline that it's going to take um there's nothing wrong with failing fast I mean like again let's talk about failing fast there's nothing wrong with failing fast but you better have a really amazingly architected deployment system and alerting system and monitoring system to be able to detect those failures so the second you move really quickly and you release and something's broken it automatically gets rolled back and so there's craftsmanship there too so again I'm not certain that these things are mutually exclusive it's just balancing these different uh different some sometimes conflicting priorities other thoughts guys I mean this is how do you guys think about craftsmanship um I was just going to make a distinction maybe between craftsmanship and Perfection maybe and that's where the distinction is you can get caught up in Perfection and and that maybe goes against a sense of urgency whereas craftsmanship can move yeah so the distinction was just for the video the distinction was there's always you know you don't need perfect you need something that's going to scale and something that's going to last and I agree I think 8020 rule is a really good rule to live by you can get like you can get 80% of the way there and that's probably typically more than submiss I guess there's also a dimension of visits on the slid so you're not talking to but I think it's important to acknowledge is like when it comes to craftsmanship of software engineering when you work at a company like when think a public company natural high high scale gr company there's a business goal and a business result right so it's not just going to L like you know art for the sake of L is actually that you want to practice your craft do a business that absolutely and sometimes the most the thing that that the solutions that are of the utmost craftsmanship are are the ones that are simple and easy right and I think again what what craftsmanship the right balance of craftsmanship with business priorities for a company like Nitro or for a venture product within LinkedIn versus you're talking about profile LinkedIn you know the bar is different absolutely but I think you you you need to start with that principle there a couple other hands I was just going to add as well I think you touched upon this speaking about technical debt you know when I think of craftsmanship and execution you can even boil it down to a craft such such as a plumber you know I can hire a plumber comes into the house at the end of the day he fixes it so my sink drains but the craftsmanship was poorly done with the next day I'm calling anothera so I think that you you know the execution could be the same I want my S working that day but how you went in and you applied your craft will will determine how long lasting that is and I so I think the two you know do go hand in hand people were talking about technical debt and I think that add something we're applying our craft will reduce technical de debt in a way that will um improve the ex totally I love that analogy that's a perfect analogy and just to borrow like an example from L my LinkedIn experience LinkedIn I think very much over indexed on speed of execution for the first you know six or seven years uh of its existence and you know in 2011 we literally took an entire quarter off from product because we gone the point where we could not reliably release code to production every two weeks literally like that's an awful awful place for a company to be especially one that you know all was thinking about going public and uh I needed to continue showing you know significant uh year-over-year and month over month growth uh and so we stopped and and literally like 400 Engineers spent all of Q4 in 2011 you know paying down technical debt that was an incredibly costly thing for the company to have to do turns out we were fortunate enough to be in a successful place where that was worthwhile but if that if that reduction of debt was just amortized over every you know product and Technology decision we had made over the Pyro 6 or 7 we wouldn't have had had to had to take on that that and for for many companies that's that's something they can't afford to do so apparently we all think about this in pretty similar terms so this is probably going to be very similar to what Steve said with maybe a couple of different words uh but my first thoughts when we were talking about value and craftsmanship is and also quickly is uh relying on our knowledge experience ability to be thoughtful about what we're doing and apply you know whatever context we have to trying to identify what parts need a bit more love like on the first pass and like what are the fragile pieces and like at the management level valuing the Judgment of the uh Engineers when they say like this is worth spending some more of my time on right now because it's going to pay off long term so you know when I see value craftsmanship I see like having appreciation for the fact that some things are going to take a little longer to do well enough and that part of that craftsmanship is identifying that that kind of attention doesn't have to be paid to every detail in what you're building but having a sense of context for where it's really going to account to the business and to the finished product totally agree I think that by the way also relates to uh to one of the other principles here and and I think um uh I'm trying to remember where I had this in the post but really you know yeah like being able to push back on arbitrary product timelines to be able to build things right the first time totally agree 100% should we move on to the promise we'll give you a chance to to ask a question later Greg maybe you can at me that great and we'll see what happens um so through touched upon sense of urgency as well as instrumenting everything right and so I think just when I combine those two myself and when I think about it is uh you want to kind of get something out there as soon as possible Right but not just for the sake of getting it out there but really kind of for testing the water right and so instrumenting I think which is kind of potentially counterintuitive because if you release something very quickly that hasn't been instrument and you've done nothing so actually you have to take even though you you want to like the faster you want to move the more you have to instrument up front totally so so so I I would I agree like these things all are very much related so so one other point before I dive in instrumentation one other point on attacking each problem with urgency I think you know it is problems in software don't happen you know randomly or by happen stance like there's always a reason and you know the nastiest problems are the ones that take the most time to isolate and we're constantly as Engineers juggling you know our current priority the current project we're working on the thing that you know just got released and like issues in production and I think that if we don't when when issues in production surface if we don't take the time and don't have urgency in addressing them we we just put them back on the back burner and they continue as being you know open bugs in our backlog you know two things happen one the quality of the product just deterior Ates pretty aggressively over time the second thing is you know the thing that you thought was a small problem you wake up you get like a a a pagro duty alert at 1: in the morning on on you know Friday like Friday Saturday night and the sight's down and more a lot of times these are things that you kind of knew about that you should have addressed and you put it on the back burner so having having the urgency to attack these things when we see them and seeing them all the way through is is super important uh to building a high quality product um as far as instrumenting I think you nailed it uh it's instrumenting not just the our you know the health of our services not just you know the the kpi that we're trying to move with a part in a particular product or a feature that we're releasing it's our build process and how long it takes it's our code coverage it's our um you know the number of of production issues we we get relative to the number of bugs that we catch in regression uh it's all of these things and and and my my real my real Point here is that we don't always have the time to do the analysis that that we get uh uh do the analysis of data that we get from instrumentation but that's okay like like having the data is so much better than not having the data and then needing it right and how the hell are we ever going to be making informed decisions about product or about technology or about our development processes without that data and add adding that instrumentation and then making sure that it is you know one of the other challenges with instrumentation is that we we have so much of it that we wind up you know having duplicate uh tracking and you know not not not resolving it and like that makes it really hard to make good good uh product uh data driven product decisions so you know the investment in instrumentation during the you know during the work the R&D workflow I guess I'd call it uh just will pay off whether or not you have the time to to look at it that's interesting right because you're making a case for um instrument first informance and collect all data even if right now you don't have the time you don't know how to correctly report upon it because once you move into the future you can never get a big data it's just so hard right like we've moved on and like then it's like always like it's what we just talked about which is like okay well you know Uh something's broken in production we don't know why we don't have the data that we need to look at it or well we really think that we should be investing in this product area that we've been touched for a while but we don't know if the feature we're thinking about building is actually worthwhile compared to the other things in our backlog and ultimately if we had that dat then we have to make a trade-off between well do we invest the time now to open to reopen this code that we haven't looked at in 6 12 18 months and put the instrumentation in place so that we can make the decision or just not do it so like it just compounds interesting and by the way I I'm not advocating not doing the analysis but like at least you know if if you're not going to yeah yeah was it's interesting that uh as relates to you know several months ago we were going through our series B fundraising and at that Adventure side technology follow over there uh Adrian Co was was a cloud architect at Netflix previously and he that was one of the things that he really stressed on during technical due diligence was about data collection instrumentation do we have a finger on the pulse of what's actually happening with our system and famously at Netflix he used to say you know Netflix is just like Cloud you know SAS service that eventually actually stream some video but but first and foremost what it does actually instruments the hell of everything happens right like most of the uh B of they shifting is actually instrumentation but to to to measure how how good the quality of R viewing experience is or in the back end it's about uh you know all the all the spunks of this world and some Logics right like looking at the actual how the code is performing and and what's happening the logs and all those things it's kind of interesting to think about like really large reactive systems like linon and and Netflix they really their first and foremost job is to like instrument itself totally you know the uh the classic quote here is uh if you don't if you don't measure it you can't fix it right and in the in the case of Netflix and and honesty LinkedIn we're not measuring how we're doing and trying to deliver the value that we're trying to deliver our members like how the hell do we know that we're even like providing any value in the first place right um and then you know the next thing you know your competitor is doing a much better job of delivering that value and you you didn't even reach realize so you know absolutely um before we move on to the next thing if there are any uh questions or comments on this one so we are we here so we here at Nitro are trying to um get into this Nitro Accelerator program and Sh was talk to you about it which is we're trying to sort of like 10x which 10x our development efforts and and and the output so we're trying to just like you know put in a performance upgrade to our teams how do we do that and then also do fewer things done better seg so so you want to go 10x right go 10x you want to instrument everything right uh you want toid crmh you want to valid execution and what of the think is them back well so so is I mean I don't have a ton of context on the things you guys are considering as you're talking about 10x it's not clear to me whether it's it's 10x everything 10x productivity 10x uh scale 10x user growth um so so that's you know it's hard for me to address this specifically I think the idea of of of 10 Xing these things absolutely can be applied to the fewer things done better principle right so if you can like take your you know take your current you know customer base or user base and if everybody in this room all they focused on was with the existing products and and with your existing products existing features existing technology just investing in that to make a 10x that is way way more valuable to your business than you know uh thinking about you know this great new feature or this great new idea because the reality is there's probably is a ton of lwh hanging fruit in what you guys has either your processes or your products or your Tech right now that you could be investing in to get that 10x uh Improvement and and I think it's doing fewer things is better really is just about uh holding ourselves like keeping ourselves honest right like as engineers and entrepreneurs we just were constantly like oh this would be amazing and if we did this and this would be amazing if we do this and uh but we only we only have finite resources right and and dog piling on on the most important things to get them in front of your customers or to get them in production or to fix you know why it take your your cut your bill times in half or whatever it is and getting that done sooner is actually much better than doing 10 things in parallel and them all taking a really long time just comment we thinking about that right now or pondering if this is unappropriated for us to do it is uh thinking about what kind of like top three to five things you want to do every single quarter and that's ass sign a dedicated thing to just that right cuz right now what's been happening is like as we're growing both in terms of users or in terms of noise or in terms of just number of engineers in the team right it's really easy to get busy with something um and then everybody does something and then what does it all amount to right and and it almost seems like some sometimes those things just take longer and longer and longer than they should because you know it's an engineer SP spending half their time to get things done while they're spending another half their time doing something else and as much as we think that we're amazing multitaskers it context witching is actually fairly unproductive so so that's exactly ironically like that's exactly what we do at LinkedIn you know quarterly quarterly boundaries are super arbitary right just like just a good way to like say what are we going to do in the next three months 90 days right pick three to five things and it doesn't matter and and there are probably other things that we may think we might want to try to do but if we don't get those three to five things done then we up so uh that is that is absolutely kind of embodying that fewer things done better principle so I totally agree you had a followup question sounded like okay um so the followup question to that was we you mentioned you're leading an organization of 150 development books right now it's about the size of of nitro so going back to going back to where you were or you are where we are right now and then let's just say we ask you like how do you kind of get more from less like what is your advice for for getting getting where you at from here right I guess following this Mantra and trying to do accelerating our accelerating our our program and our capabilities it's it's a hard question again without spending a lot of time with you guys um so so so may maybe we can WR because you kind of need lot lot of context to answer this and and and we're trying to keep it brief but maybe if you can just reflect about if there was in a hot moment or like a big lesson learn when you were going for maybe like 30 people to 100 people that type of a growth at that time we kind of like you need managers manages you got multiple people working on multiple things it's usually been most startups right it's like that magical Dumar number you know an organization cross that kind of human capacity just know everybody understand what they're working on like now have to break up yeah um it's it's I don't know that there is again like a because every team and Company is different there's no in my opinion there's no pattern that you just apply to that just works everywhere um I think that I think you start with understanding my my sense is like things are obviously working for you guys right now like spend the time to understand what what it is that's gotten you as far as you are like to to this to this point those things are going to scale like spending some time thinking about how you scale those things when you go from 30 100 is important so for example if you're talking about now we actually need to put some more organizational structure in place and we need to hire managers that's fine you know those managers should be very much you know everything that they do should be should be you know within the context of those three five 10 things that got you here today and reapplying what got you from you know your first engineer to 30 to their own teams right and then I think that there is some you know processyoutube So Pro so so things actually move faster um but I think some of these principles you know again for us were foundational and things that we carried through all of that growth and every new manager came on board you know embodying these principles and applied them to their team and as they developed and and and coached new Engineers coming onto the team they coach them in these things and that's kind of what carried us and and I would you know it may not be these things it might be some probably other things you just figure out what those things are cool so we have um less than 10 minutes um and I think we already slightly touched upon prioritization and planning um but what we haven't talked at all is the hot horing bar like putting the them together right yeah um so again you know I think we've been recently uh historically we've been say hiring like a pace of like one person per month and the last 6 months we've gotten it all the way up to like five people per month so like like now going through that phase of like faster growth and faster hiring so again just kind of going off your own experience um when it comes to establishing and and hopefully even raising a car and bar how you've done that over like going from like 10ish to more than 100 engineers in the team there's this tool called LinkedIn where you can find Engineers yeah he actually paid him to set me up with that question so that's right uh yeah you get a free subscription that you can use for for your recurring needs you know what like you know I love linkedin's kind of expens so like I'll appreciate okay perfect um I walked into that one um it's recorded So I I think that you you scale again I think recruiting is like anything else like imagine this was a software problem you were trying to solve like you instrument recruiting and you figure out uh what parts of your pipeline and your interview process is working and what's not like I think at different stages at LinkedIn we had different problems sometimes you know top of the funnel you know we weren't getting enough people on top of the funnel sometimes we weren't closing people sometimes we were losing them to Google sometimes we losing Facebook like we we we've what's Google what's Google uh yeah they're I don't know I've heard of them I think they're like a search engine or something um they uh I think they they they grow Engineers on trees rather than they like they've instrumented the point where they you know they that's right um so I think that's that's a big deal I think that you cannot you know I can't stress this enough you know this sounds super super cliche to to say oh you got to keep a high hiring bar and you know you don't B the hiring bar but like you know we've all been in these conversations where it's like damn it like we like we need five more Engineers to get this stuff done that's super super important and like yeah you know you know he wasn't like kind of worried about how he's going to get along with the rest of the team but like he nailed that design like that that software architecture problem right and and or he crushed the coding exercise ROM do um and there's all so much pressure to hire so that we can get stuff done uh and it never like that always fit like always always always fails and so never compromise never like you you just have to have the again you guys have built an amazing thing here uh you built it mostly because of the caliber of the people that you've invited into your circle here right uh and the thing that will that will hurt you more than anything else on this list honestly is is starting to grow and as you talked about scaling so hard right like you know it's very easy you you guys all know what the Nitro culture is like and you all know you know all this tribal knowledge and all these things and as you scale uh starting to inject you know bits into this system that that you know aren't going to operate as well as the current bits do is is going to be pretty D destructive so again super cliche something that we all talk about probably every single one of you guys talk about and yet In the Heat of the Moment in the moment where you're making those those you know interview feedback decisions or that you're actually as a manager or or as a team you know reflecting on the interview feedback uh that's the time where you have to really reflect and hold that bar that that that bar super high Greg's got his his hand you going to finish on a high note if I may just bring it back to the first one uh is I think you just described the craftsmans except for it's in recruiting yeah the moment you focus on execution of hiring over the craft of like that U evaluating talent that's when you um feel just as much as you fail and you put in a piece of code that you just kind of pack and then sh spot on and it's not like totally right in fact we could I we could completely eliminate this last bullet and just apply like you said that the the principle of craftsmanship to everything that we do as engineers and it's it's not just building software it's recruiting um and I think we could probably you know brainstorm a few other things that we're doing dayto day where like the craftsmanship principle applies and again it's not just in the you know the recruiting process but it's in the things we talked about instrumentation and are the tools you guys are using are you guys happy with the tools that you guys are using for interviews like we used jaob forever and it sucked and it was awful and we killed it and we built our own um uh five six different tools so like that's super painful too and it's costly too so like again then it's prioritization like we want to apply craftsmanship to the tools that we use in recruiting how do we like at some point it's going to pay off because you guys are you said you're like 5x in the amount of interviews you're doing now because you have to those recruiting that's a lot a lot of time and a lot of frustration that you know the folks in this room are probably enduring using those crappy crappy interview tools I've I've been there so yeah I think we've done in the last like 3 six months we've done a big upgrade on both the inter experience externally and internally for for for our team and that absolutely out there instrumented shared instrumentation to the team fose to the top of the funnel and kind of drove it down and actually apply different tools and practices like at each level of the funnel and have a James K I don't know if he's here feel like that's amazing for of exactly where things are and how we're doing and then we know where fol nice well gr you get you get double takes because you invited there than that was actually my original question when you guys Ki me but um but uh the well so I guess we're talking about crmh like one thing that didn't touch on which is important is also like having pride in your work which is not necessarily always measurable or instrument instrument but it's just a sense of cohesiveness and pride what you do and you know building block again for your IR team so forth but the question out of that is um like this these slides are great and these the points are great but you touch most on like the technical aspect of it or like the meable ASP what about things like emotional leadership in the team or um that aspect of well I mean because team building what does that fit on this um on this um can you elaborate what do you mean exactly by emotional you mean like emotional I mean you can like I guess it I guess we sort of touched upon it with the hiring bar but um you can have everything instrumented and you know execution but if there if the some a lot of teams like are you talking about cultural values or um I mean are you talking about so so so is it I I think I think that there is let let me try to answer that question cuz I think I know where you're getting at like there is there to to every part of this there's gut right like you know when you think about data driven product development there is always you always have incomplete data and so there's always your your intuition about the right way or what the right thing is to do based on your experience based on your your know knowledge Etc and and uh the reality is that all the decisions that we're making across all of these principles and other things that we've talked about the application of these principles is is applying the data that you have and like the rigor that you that you put into getting that data and what your experience tells you uh if you're not if if you're doing too much of one or the other you're probably not making the right decisions and so in the case of hiring you know super in like uh inexact science right like it's awful like you're you know I I will say that you really really are better off optimizing for for precision there right as opposed to to recall but the uh sometimes you you you go with your gut like like very much like I feel like this person would be like there's I can't can't put my finger on it they nailed everything everybody gave him a pos positive review on the culture interview and yet I just don't feel like he's going to fit in and I think again in this particular case optimizing for precision and and not having uh a false positive is better than than having false negatives cool all right so we right up 1:00 um I know you got you know this company work for um so I I'll do one quick pitch by the way uh not for our LinkedIn recruiter product which I already did already um so we we have an office here uh at 55 Howard Street first in Howard which is like five or six blocks from here and we do monthly Tech talks and um about a variety of topics like we recently had one about data driven product development which sounds like is is really core to your guys's culture we had uh some really amazing speakers from Pinterest uh and airbnbs uh as well as our data science teams talking about how how we apply data driven development earlier in the year we had some folks from the uh uh kind of the VC World talking about technology Trends we've had uh uh talks about uh client side Technologies Etc um definitely invite you guys to join like part of what we part we our office is now like you know six months old and and and part of our you know our our uh thoughts in in creating the office was to provide us you know linkedin's mission is to make you know the world's professionals more productive and successful we feel like being in the city now and being in this technology Community we can use our space uh and our Network to help engineers and people in technology be more productive and successful by having them share their knowledge and their insights with each other to the extent that you guys have amazing things that you guys want to present um I would love to hear it to the extent that you're just interested I can forward along kind of the uh the uh DL for for signing up for the tech talks thank you very much but be careful who you like and what you wish for uh be beer fridge or whatever else is there perfect just a bunch of hting badges lot of um so so thank you very much thank you guys very much