DBTB INT Eugene Kirpichov r

so my name is Eugene Gorbachev and i'm a senior software engineer at Google working on cloud dataflow and apache bean so I'm absolutely in love with data processing and with beautiful programming abstractions and with my team so for me this is a good chance to share that love and share my fascination with a programming model that has come out of many years of work of our team so I'm hoping to get people excited about that I think this model is really beautiful I'm hoping that it will have a similar effect on the data processing industry as the MapReduce paper ones did also since the bean project is open-source I'm also excited to invite people to join its ecosystem yeah these are the main exciting parts so it's a big honor and bigger accessibility to be here and make sure that I share all these ideas in ways that stick so to me the most exciting part about data is the techniques of processing it since that's what I'm working on and to me it's fascinating how difficult it is to do everything right in a distributed data processing system for example the project that I mainly working on we just published a blog post about it yesterday it's called no sharp left behind it's about dynamic worker balancing and data flow when I joined the team and I was totally starting this project I was wondering why isn't everybody doing this it seems so simple now three years later I know why so it's very interesting how much you can keep pushing the boundaries of data processing how many challenges there are and how interesting they are to solve and how interesting it is to debug all this so I think this is one of them are most interesting areas of software engineering making this kind of systems I think that would be to never settle when asking for more from your programming abstractions they've been programming model itself is the result of multiple steps of that first refusing to settle for the low-level API of MapReduce then refusing to settle for approximate results when computing our streams then refusing to settle for writing different code when processing batch and streaming then refusing to settle for having to deploy and manage your infrastructure the data flow and finally Apache beam is the result of also refusing settle for looking into one platform because it runs over any platform I think again I'm going to switch this to the question of becoming a data processing engine developer because that's what most close to me for me the path was relatively straightforward i just got fascinated with this topic and i kept learning more about it from blogs and papers published by various companies dealing with big data including Google Facebook LinkedIn Twitter and others so I kept learning their blogs and papers and kept learning more and getting more excited and eventually I learned enough to end up with this team