sfspark.org: Peng Ye (Aerosolve) Interview with Alexy Khrabrov
Recording: sfspark.org: Peng Ye (Aerosolve) Interview with Alexy Khrabrov
[Music] all right hello everybody I'm Alexi kov the organizer of SF spark and here we are on location at hiroku uh our Co organizer Simon Chan of prediction IO uh acquired by Salesforce uh gracefully uh hosted us here and this is our first time in is the beautiful office and here we are with panga she's a software engineer at Airbnb and she's going to be doing a really interesting talk about machine learning and we're very happy to have her welcome uh thank you for inviting me uh tell us a little bit about uh uh the system we're going to talk today and how you got interested in it and uh how it happened how it was created at Airbnb okay so today I'm going to talk about uh AOL AOL is open source machine learning library that was started in the pricing team at Airbnb so actually the architect of Airsoft is Hector so that project was started to primarily uh support the pricing modeling problem so when I joined the Airbnb uh the like the Prototype of AOL is already there so I basically picked up everything and like start to do modeling work using ourself mhm so uh it's it's interesting to see more and more companies using uh machine learning rbnb is at huge scale so how does the fact that rbnb is so big kind of motivate the need for something like aerosol yes I think the problem is uh there are a few reasons for us to to build airsa by ourself first we have really huge amount of data so for company at the schale of Airbnb we have a lot of data so that's why we need something to allow us to do distributed training uh like on a huge amount of data like in this case we choose to implement this using spark so that's airof that's why we need airsof machine learning yeah okay and uh I mean obviously spark is uh really popular now for um uh distribut systems for machine learning uh I know you guys used mesos uh for a long time technical talks uh right and there is Kronos at rbnb and I know some folks you know who wrote Kronos moved on to mesosphere uh but so I'm really curious how do you guys uh choose spark like why you know not something else why not something else so I feel this is like a easy option right so we don't have any better alternative to build the like M huge large machine learning model right okay do we have any other option do have anything in mind uh it's it's hard it's hard to find uh really in in you know something comparable to spark right so it's really it's really really it's really good to to do spark so basically so spark came about as a good choice for for for the distributed uh part of of that so spark how do you use spark do you use it with python with Scala uh how do you write your your code uh we use spark with Scala okay so basically the AR of training code was all implemented in Scala mhm yeah nice so how do you define Scala for data science and data mining how do I find scholar so when I started at Airbnb uh it the airsoft is in scholar that's that's when I started to learn scholar yes so actually before join Airbnb I I didn't actually use Scala okay because I was primarily in the computer vision uh field so where people don't use Scala like for modeling interesting so so so once you started using it how do you compare it with you know previous languages you used previously I was doing deep learning at Sony where I use the python and I use like seano stuff uh so python is really easy to use right so but Scala I feel Scala is more functional right so uh initially I don't feel so comfortable writing Scala but once you get used to it I feel yeah it's a beautiful language how do you find kind of the whole cycle because it's compiled right it's you know it's CU rapple do you find that it's fast enough for you you know do you find yourself a lot of time in the reple or you mostly compile it and run the whole program like how's your day kind of with Scala how's my day with Scola so so okay so usually how do I I will describe how do I like write code for aoft so mostly I will like write my codes but most of the time writing code and then I compile so Bas I compile the whole jar and then I upload that that jar to some spark cluster and then try to like train the model and like tune the experiments so I don't feel I'm like spending a lot of time just on compiling think I was mostly working on the actual coding and the like the experimentation nice so compiler speed is not really yeah it's not something yeah how about interactive rapple you know a lot of people basically like sculla because you know python gives you uh prompt right and Ruby and people kind of expecting that and scull also can give you that did you find yourself using interactive mode in Scala no actually not okay that's that can be something interesting to to experiment I think spark is a local mode and you can probably a lot of things yes so actually so the way we do is we we write unit test right so we can like test locally but often times for machine learning you need to actually test on the real data so that's why I often so most of my time spending on like like uh actually like uploading the the model to the the real cluster and experimenting there MH on real data yeah nice nice and so you doing computer vision before using deep learning so uh uh basically how do you find kind of the uh optimization problems with there BNB in like what what's in common with you know the field Vision or machine learning in general yeah so I think uh in terms of what's in common so there these are all machine learning problems so mostly we're all all dealing with different types of classification problem regression problem M but I think it's very interesting to like start to work on the pricing problem for me because my most previously I was primar working on computer vision where I'm mostly working on the dense feature so the feature itself is really not interpretable like the a row image pixel itself does not make sense but here at Airbnb I start I worked mostly on the interpretable and sparse feature so I think the the types of feature are quite different for the pricing compared to the computer vision problem interesting and how do you see kind of your work being used do you see real results of these models yeah of course so I worked on building like one version of the main pricing model so our pricing model is trying to predict like booking probability and then from there we provide price tip to our host so so the currently the model I train is actually in production so the pricing tip that we provided to our host actually generated by this model mhm so and from our a testing so the result is actually quite like it's quite good so we significantly boosted the the night's revenue for our host oh nice yeah that's interesting because I was booking you know U stain Venus for my family for the next year I probably was using you know some of the prices by a model yeah yeah it's possible yeah no this is great so thank you very much we're really happy you know to have you here and we're looking forward to your talk thank you thank [Music] you