Debora Donato: Q&A with Alexy Khrabrov of SF Scala @StumbleUpon 20150122
Recording: Debora Donato: Q&A with Alexy Khrabrov of SF Scala @StumbleUpon 20150122
uh hello everybody uh I'm Alexi kraber the organizer of SF schola and we are on location at stumbleon where we are uh having a meet up tonight uh about prediction iio and stumble upon itself and I'm here with deato the senior director of engineering and principal data scientists stumble upon uh so this is subsolar developer series and uh we ask folks uh about their uh profession about their tools about the companies uh so I'm going just start uh with a question uh so what is uh what stapon is doing uh and what kind of problems are you solving stapon yeah so I will talk this a little bit you know in in my in my talk but uh someon is uh trying to address a problem of CP search that is basically discovering new content uh often people uh misinterpret this as another Google or another Facebook but actually the problem space is totally different we are not search we are not another social network but what we are trying to address is the problem of convey information or content to people when this people is not really looking for something in particular but as a broad interest uh there are few new startup that trying to address the same problem but somewhere upon it's in this space since more than 10 years so one of the oldest startups in the cicon valley yeah and so to is not a startup and so we we extensively have study all the problem related to Discovery or sity search mhm uh so uh uh I think your your title is very interesting because in many organizations data science and engineering are separate things so I wonder uh what do these things actually mean to you especially data science because there is a lot of different meanings people uh uh add there and how does it work for you to to kind of join these two two things okay I I come from research so originally my background is a research and apply the science and one of the main problem for researcher or appli scientist is that the research pro project doesn't go anywhere if it's not B up from engineering engineering is the true backbone of innovation uh and in many year I got you know frustrated because my project were stuck somewhere because there was this boundary between science data that became data science when I started was just a science uh and and Engineering so stumbleon gave me the possibility of basically fill the Gap so and finally have the possibility to enable machine learning a scale and this cannot happen without you know engineering and science working together and so for me is a huge opportunity you know to to finally start a project from the very beginning and bring it back to in front of the user uh and and I have to say this in my opinion is the only way in which data science can actually really work within company having something that is totally separated from the engineering organization in the company is not something that is sustainable in in the long term okay uh so but you know like at web scale everything becomes big data right which is another uh IL defined term but generally it may mean you know uh web scale platforms which react in in in in in near real time to a lot of clients with apis and then and then big data processing and there so there is all these questions of scale and devops which is usually covered by a platform so I wonder you know uh how do you manage to address all of this is it kind of within your uh area of responsibility if you faced with problem of scale you know and kind of uh hardcore uh engineering problems going with this okay this is an interesting question because so there are problem real problem and there are I would say fake problem in the sense that uh the pro the product that I inherited the I I'm leading the personalization uh since six months uh was after all an offline bch like uh product what I mean with this a lot of computation is not done in real time a lot of computation is done by Crown job batch and what you actually do in real time is just serving uh recommendation through elastic search or through any you know uh kind Lucan based uh this is actually is not a problem of scaling in the sense that is something that is perfectly addressable with without you know uh considering performances the other question that people ask me Big Data how do you train model again this is again not a a problem in the sense that training a model is not a a problem of scalability it's something that you do offline you sample uh the real problem in my opinion is uh enabling thata science and this come a step back enabling data science means doing smart data process processing MH uh and this smart data processing was not in place uh when I took over the grou MH uh so what I I immediately realized is that to bring the product where I want the product to be uh I really need to start considering how I can in real time uh take all the inputs that the web produce and this is not just rating and put this in a model that real time basically helped me to uh improve my product so for for this just this part that I'm telling you data pre data processing and creating modeling around the data to better understand user Behavior yes I needed to take in consideration prob like scalability uh realtime computation and here is when I was let's say introduced to Scala mhm and I I am lucky because we have a platform team that are David that was that really introduced me to Scala and help me to basically understand what are the limitation of the current you know pipeline that we have and how to address this limitation in I would say a more reactive and dynamic system uh and uh this is what we are actually trying trying to do right now okay that sounds great and actually this is interesting you know I've made this observation recently I was at next ml uh one day conference and a lot of data scientists are using uh different kind of tools for instance they're using python they're using theano they're using Lua so they want interactive for Apple they want to be in this environment where they can do Ada uh but they want performance so all these things compil down to C right and they still stay within uh single node maybe multicore maybe GPU but they're not distributed right so so scholar gives the Apple it also is backed by distributed systems such as Spark all right and but you know some of the libraries are missing you know pandas some some stuff some are you know uh are people like all their packages so I wonder what's your take now that you know you've seen the scull ecosystem uh what is the potential for data scientists kind of coming to scalar platform and leveraging the distributed component you know what can we do as a community to bring all the tools or most of the tools which people enjoy in python or R is the is this a problem for you for instance that you know some of them are not there what do you think of future of data science on scholar ah I have to say so data scientists usually are not engineer this is I will say it's the big problem and so they rely on tools like R uh or you know python as you as you mentioned um I I do believe that if if we just focus on data science like an exercise this tool are good enough you know you can model uh your problem and you can train your classifier uh but again uh to bring data science in your product I really don't think that you know the current tools are performant enough uh and I I really believe that there is more and more interest from a data science to basically see uh what what they working they model to be actually used in Real Environment uh and so um I have to admit assemble upon was not use of spark there was not use of uh not even you know I say mouth for example because what we were trying to do is just this batch working but uh again now if you want to be competitive you cannot rely on the computation that happens in a single node I mean you really need to model user Behavior real time uh when more and more people will realize that this is the next step you really enable machine learning enable future architecture enable um you know a series of of Subs system that are around for example recommendation uh I was thinking that the Gap would be filled immediately uh again for me was like working on data just on data science I I didn't have the perception but this became immediate as soon I took over the personalization grow so that there was a gap to fill and so here the interest of personal interest and my group to uh what is available prediction of your spark storm so we are deeply modifying uh our infrastructure and our architecture to enabling data science so I I do believe that uh with this passionate community and you know with the ability also because data science is permeating now many many aspect of different component in different corporate I really believe that it will just be matter of time before we start seeing more and more uh algorithm and um I was say implemented directly SC it's great to hear I mean that's kind of conference you know my intuition so it's great to hear it from a practitioner and you know the this is will be one of the themes of our big data scholar conference in August where we essentially we want to POS it as a challenge to to The Scholar community that you know we have the the computational means right let's kind of fill in the gaps so that beginning data scientist can be taught this and maybe they will start from much stronger power so another question I want to ask you you I noticed that you you you come from European tradition and you know I also come from European tradition of you know math and physics and I wonder uh how does this affect right your uh kind of your your view uh in in a startup culture which is right so academic tradition is about rigorous you know build up from the first principles and and and scientific approach and proof and and so forth right and and convincing your colleagues that you're right uh and here basically it's a business where you convince people that you know you're right if you have more clicks and more visits and and stuff like that and so I wonder how do you find you know these kind of two cultures interact in in in Silicon valy from your point of view ah this is a very interesting question so it's true I I found that uh my background is different so in the sense that uh for example the way the way I act I like to build a product I I like to believe that we can make a difference in a product doesn't matter how many click your product has In This Moment uh and I mean if you think about it this is kind of really really different if you think in Europe uh people start a job and probably retire after 20 year with within the same company for 20 years and and I I really believe that not anything so this the startup idea that everything can be created in uh I don't know in a few months and you can collect million of dollar and pass to the N startup probably influence me I I have kind of say love of the product that I working on I try to be passionate about the product I try to uh you know to to to always think long term where where I want these companies to be in 5 year in 10 year and what I can do to bring this company to be there in 5 year and 10 year uh and and so I would say that this is basically different from the startup culture in which you just and and this is one of my I will say my probably uh I not problem but the the way I approach you know the problem itself uh I I don't uh I usually do not plan for the small the the short term or the immediate reward uh I do believe that you can accept uh The Click rate going down for a certain period if what you are planning for is an increase in the product if you really point I mean Innovation doesn't happen in in a night Innovation does not happen in you know in a couple of months and and I do believe that also it's not the money that you collect that actually make your product strong uh and so I have to say yes I I'm I will not Define myself an entrepreneur I'm not the startup type but I do like the mentality the agile mentality I do like the fact that you can cultivate a long-term Vision just but with you know rapid uh step rap rapid pace and and this actually something that I learn and I start you know to appreciate when I moved in Silicon Valley oh that's great so in you know then maybe like when you know to your point what you you see the uh Horizons so maybe you will end with kind of the prediction you know because you know you plan five where do you want stumble upon to be in five or 10 years and what do you personally want to accomplish you know doing it right like what are your kind of challenges for yourself you know for the next few years yeah so for me uh where okay let's start where stumbleon want to be uh we we we are ambitious we want to be the first uh content personalized content engine in the world uh and I think we we we have the expertise to be there uh what I needed to do for me to to be there uh is to basically address the limitation that I currently see uh so there is a lot of learning that I started when I took over the position of of Senor director of engineering uh fill the gap between data science and engineering and this in my opinion is extremely important to bring you know to to meet to meet the the longterm objective for St uh personally from my perspective is a lot of learning both from uh I will say the data science perspective because I don't know if you're familiar with the recommendation system but is a field that I mean you know algor and uh uh methodology there is one new every single day if you follow rexis this year was uh uh extremely extremely inspiring our other you know uh conference like kdd this year with uh data science for good was another inspiring moment so a lot of learning just on you know science Direction machine learning Direction but also a lot of learning uh for me as engineer to you know to to help me to fill this Gap and to work in this distributed setting that was not you know in my core uh since I come from from science uh but I I I really believe in this product and I think we can get the the goal that sounds great so I hope you know we'll we'll revisit it in a few years and see you know how it turns out so thank you very much and looking forward to your talk thanks