sfspark.org: Alex Oleinikov Interview
Recording: sfspark.org: Alex Oleinikov Interview
hello everybody I'm Alexa crabbers the organizing process spark and here on location at Chara boost at our meetup about sparking a deck and we have Alex from tricky Oh so Alex Elena Kelsey the software engineer at creative working smart welcome Alex hi so can you tell us a little bit about Christian org righty-o why do people say it differently and what you do there and what created us all right so criteria or critter is pretty big a tech company one of the unicorns using corn startups and friends pretty well-known except that in us maybe not so what we do is a bunch of different stuff had president kale for instance who do are to be or we do retargeting and for that half huge infer for to give you some numbers for reference we serve like 30 billion requests in our are to be per day and we have say 20 30 hundreds of nodes in our header clusters so we do stuff at scale and parts of the complexity comes with that and we also have slower projects which are longer shots not necessarily the same scale but much more agile one of them is search which stands for optimizing search engine target them search engine required hmm so yeah that's what I'm working water for we're cool so for those hoops I knew to attack you know what does all this RTVE maybe the exponent about the challenges on all right so our to be as a mechanism which stands for real-time bidding is a thing when you're a public or publisher just about to display the ad there is a saint has happening on the background which basically falls all the possible people who want to display the ad and ask okay here's the users here is the patient which I want to display the ad and how much you want to pay great for me right you pay me to display their right and that happens in about like 40 30 we have to be super quick with your decisions and that's what people do I was really surprised when I learned about it maybe a year ago I was you know thinking like a no-return like a piece of booty from the ground I open it up and we Romans bar there right so this is like you don't normally go to a website right yeah meeting is coming at once like all this you know living mechanisms behind it so so so this whole area of all right so one of the reasons we do this it up again I want to showcase their tech industry right there Intel engineers more about it because a lot of 50 years of this group will do traditional systems mostly you know 80 is and kind of data bottles right so what little areas very fascinating so how did you get interested in in effect well first the problem of course I guess the the most complex thing in attack is the amount of data we have to crunch because like POV indices have that amount of data because in that sack you can hold like all the huge amounts of those tiny events which make up the whole difference and we have to crunch down at big scale so that's what attracted me in the first place so having like having to played without all that interest so with problems with no space for making any mistakes and stuff like that and plus I was processing the data and extract and knowledge in my previous job so that was kind of that was kind of a natural transition for me so you know there is a real effective school a lot of time spark or so right so yeah how so how does this work and you guys can excel to be powerful spark even despite what richness to this all right so historically in Krita we use Hadoop for applying processing and we slowly replacement with spark so to serve those queries you obviously have to use machine learning and obviously machine learning has it's an offline part so that's what the offline processing is used for in Krita in general again certainly we have had it there but for the new projects or for the refactoring of the nickel we have sparked and that's pretty much what we do plus as I tell you we work with search engine advertising so that we're that's like that's area in which the performance requirements are even hires that Google Google does not allow you to do the RTD then it does you bid offline so you have your own modifiers you have your own bid and if you want to do it offline obviously a spark as a good fit because there you don't have such strict requirements on your site all there's not a fulfilled by Google right very interesting and so look at Cree do engineers speaking early in our numbers are so not just about any others and you know there are various projects around it all silence barn like open source systems creative also how could you guys evolve such an interesting in genealogy people's tools how does it work how all of the knowledge is presence at the company if you can tell a little bit about engineering culture all right so I think every cup every group including the company is pretty much on its own with this so of course we have technologies which are kind of promoted but you have no you have no means to force people use technologies so you just come up with whatever fits better with their problems and then when we have critical mass of people which use the same thing and they have shared knowledge we have a concept of guilt but also guilt are basically groups of people who are interested in the same topics and they want to share the knowledge that you want to promote and and when people can come in and they ask questions they always have you know the authority to answer them that's the way it works do you have any message spread the references somebody wants to learn more about sparked the five you over-aggressive a new class others work well again it very like it depends a lot on the on the team you're working with because like there are no strict rules for instance some teams they do tactics some within the company some hints to Tech Talks even within project there are initiatives of initiatives on the company's level but mostly the knowledge spreading is going on by individuals who are interested in that so that's exactly what you told what you told me before like you're just into knowledge sharing if you if you are then you just do it and some people take their time and they do it cool in the company is forces yes yeah look at this this is what I cannot have seen you know with a boxer game but it's very center because we got some hopes kind of level Paris and you guys start off as a father so that what they told us and discover that creative appreciates the biggest company concerned the bevels they are teach to the electronics right whether we just don't believe that Chris here is the French and we're real creative is called Lydia we're one of the few companies who do not know how to promote Italy so follow how does it work if I like you know in Paris and for all them to do the world's to different cultures and I love this idea well the thirst of discuss express from all over Europe right so you come with a real opportunity appears there so how how do these two pieces work together look engineering both occasions is kind of a typical girl is the great or big date innovation like we we see Stockholm from here so can just distributed common works so the way it works now is that historically of course we have all the core engineering in Paris but recently we started hit the limit so that we cannot hire even more people in Paris so that's why we start to hire in other places and one of the biggest places in is our office here in Palo Alto the way the engineer works work is split and all the infrastructure except for the local one of course is done in Paris all the tools are done in Paris and all the big historical projects I was talking about were done in Paris in Palo Alto we have things which are detachable from that like research project I'm working on we still share the same thing from we still have to communicate with parents on you know very big topics but at the same time we are we're pretty compact in terms of the amount of people we have each of those projects anything that we can communicate efficiently within the project since we are all in the same place and all the you know all the communication between people across the ocean and it's kind of minimized quote so I think we learned what about the edge of the day so really looking forward to technical talk maybe kind of will wrap up with the question and usually both here so so what do you kind of want to accomplish in the next year in the open source community experience right what do you so you here at the MIT up what do you think is most interesting for you to learn right like you know pulleys and organize or what would you like to see you know what kind of topics with you benefit from it with my speakers to speak on the subjects all right so that's a very very good time for this question because just I tell you I looked into the area like six weeks ago reading you and I would just like to use the fact that we have a very diverse community here and you have can hear about all these different topics so I would just try to prove in my novel shall be of technical landscape mm-hmm specific technologist a spark of methods like anything which can fit some the doctor mind [Music] well image processing is interesting for me natural text natural language processing is interesting apply machine learning at scale all that cannot lead to self driving cars that's probably one of the reasons why the topics the topics is so sexy okay I'll stuff like that also so I think we're going to look at what is the Southern Conference of our derivative track on this November I think you got too much processing you got ood so you know I'm pretty sure the cameras start pretty early place alright okay alright welcome handling for physical sure thank you thanks [Music]