scale.bythebay.io: Remy DeCausemaker Interview
Recording: scale.bythebay.io: Remy DeCausemaker Interview
[Music] my name is Remy to cause maker and I'm the open source program manager here at Twitter all of them I guess I'm still like learning all of the massive ecosystem of very cool projects at scale that we run here our github repo for Twitter alone just has over a hundred and fifty repos in it that's not including a number of other github repositories and organizations that all have dozens more projects it's incredible to be operating at the scale that we do but the engineering problems that I'm looking at our much less infrastructure and more community infrastructure so like the way I think about what we do with an open-source program is more like if you have DevOps which is automating all the infrastructure things I think about community operations or comm ops so how do we automate and facilitate the kinds of stuff that you do to build an environment that this that helps to build awesome projects I would say that you know there's an old saying right I'm a big Linux guy so they say Linus doesn't scale right so you have project leaders or community leaders who are really driving as sort of like a PDF l or you know core maintainer and those folks a lot of process blocks on them and you have this fundamental challenge where you have to have people to choose between building things and building communities that build things and people don't scale nearly as well as technology and tools so coming up with ways to easily onboard people make them feel comfortable and welcome to help to minimize the amount of sort of bad apples in the bunch those are all like standard problems that as you scale up your community that might have started as like this wonderful little garden full of great people as the internet finds you you know at scale like the big numbers game you're gonna attract all different kinds of attention and problems and you know you got to make sure that when you build your community from the start that you're thinking about and considering those things just like when you're thinking about scale when you're a small scale you know you want to be planning for a larger network topology is in bigger data centers and multi-tenancy stuff I mean all of that sort of placed true as well with the people that run the project I'm really concerned with the carrying capacity of the ecosystem I believe that there's a real danger in the enterprise that if we aren't finding the places of commonality where we can develop sort of commodity level software similar to what Android has done or the Linux kernel is done there's a danger that we'll see a lot of over differentiation is the term where people are going deeper and deeper down in the stack to differentiate their products and as you get deeper and deeper to core engineering pieces it cost more and more money to do that because you have to innovate at higher and higher levels on top of it anyway so the more that we can standardize the different layers of infrastructure the better it is for people who have limited engineering budgets and limited cycles the other thing that's from an enterprise perspective from a community perspective I mean there's only so many hackers right and if there's 12 forks of your latest and greatest orchestration framework or database or whatever there's a lot of duplicated work and there isn't really a sort of un of open source if you will where there's it's more of a nation-state model where different companies or organizations or communities are thinking about it very much from their personal interest in this whereas if we look more at the ecosystem and we said hey like in Linux Astroland there's a really cool project that I got to work on with the Fedora project called release - monitoring org and it was a just a map a simple map of here's off stream source repo and then here's what the command is to install it on each of the distros and then here are the web hooks they hook into it so you can get a real-time messaging infrastructure to say upstream source has shipped a new version kick off your tests kick off your tasks and see if you can do a scratch build and if you can hey maybe we can run those packaging scripts automatically so fedora was checking it out and debian was checking it out and we were using that tool to help reduce the amount of duplicate work in packaging you know I'm hopeful that we can see those kinds of collaborations across communities start to spread out into other areas as well so functional programmers are wizards their brains are they tick a little bit differently when you think in a lisp or when you can do recursion just as a second nature it's really special and working with those kinds of people is really inspiring the way I describe my technical background is I'm more of a code rogue than a code wizard I just have a really high use magic device skill so I understand very well like how to be bold and you know following the trace backs down the rabbit hole and I try to be agnostic diagnostic about the technologies that I use an experiment with so I've delved into some corners of functional but my weapon of choice has been Python and coming to Twitter has been really interesting because you know we're huge Scala shop and meeting with lots of Emacs hackers and you know getting more introduced I think for me to get more into the mindset of functional I might have to try like the Viper mode with Emacs maybe see if I could start messing with some plugins like I think that's how it started for me is just like a hacker in general was once you start hacking on the tools then you get to be more comfortable with the mindset of the engineering prowess there's just a ton of really brilliant people here to learn from as well so you know I'm looking forward to learning more about it but at first one thing I did learn right off the bat which was really cool I was like yeah I don't know what all the rage is about this functional programming stuff until I got to the enterprise and then when I got to Twitter it was like okay well if you start off with like functional programming you know what state is gonna be and when you're using microservices there's just this obvious connection where you're like I can hot swap out instances I can blow away different things in the infrastructure and it doesn't matter I don't have to reproduce state and environment when I'm testing and all of a sudden it just clicked and I was like holy like functional programming is what makes all this microservices stuff really to go and that was like a breakthrough moment and that was like in my first 24 hours of starting orientation here so from there I've been very curious about everything [Music] I'm a community guy right so I love to see the overlaps where different hackers are it's really cool I'm helping to run the unconference track which is a cool experiment I've been working with Travis Brown and John Freddie who both have been awesome in helping to get the event together and when you're sitting there and you're looking at somebody's like config for SBT and somebody had just presented like Cersei library and there it is like Cersei is in the dependencies and it's like being built and you get to see like upstream hackers talking to each other and doing like real consider it could like consider it conversations about where the direction of their projects are going asking for honest feedback there's real diffusion of ideas and any time you can get upstream community members particularly at like the language level that's amazing like you see I go to a lot of conferences that are about like web dev or Enterprise lis stuff but like really language centric stuff like the conversations you get into are just madhouse ridiculous awesome like people you know waxing philosophical about algebras and like really deep mathematical concepts and talking about like the guts of you know bas and Fortran stuff that they're porting and like really really low-level stuff that the conversations and the content that you get to see here are unlike other conferences I would say it's like extra wizard that's probably the best part about scalable