Cognifest NYC 2017: Samantha Hansen, From prototyping to deploying, building ML systems with Featran
[Music] all right hi I am Sammy and now that you know what feet Ron is I'm going to be talking about an application of it and using it in practice and pictured below are some of the people on Homer that I've been working with Karl James boob Ben and Tiffany okay so again now that you know what feet rent is I'm going to be talking about an application of it and so the goal is for you to understand how to use VTrans for model training and also in a service that evaluates the model and in order to accomplish that I'm going to take you step-by-step through conception to deployment for an ml inspired UK use case that's modified so I'm using the word inspired because I'm not going to go into all of the details of working on it so basically cutting out some of the extra stuff in order to make btran the main focus of the talk and to let it be the true star of this presentation and a tentative outline for what I'll be going over is first defining the problem next talking about an ml solution for the problem going into feature and model selection and finally into the infrastructure that we are building and had built to accomplish this okay so starting off with the use a stand ml solution so from the introduction I work on a team called Homer and based off of that name you might be able to tell that we work on the homepage for Spotify and the use case is that we want to order content on home and so currently home is structured in the following way we have a shelf which is characterized by its name such as you see here it's made for you and you might also like along with the collection of media items and so a media item is defined by its URI and that's basically a link that you can click on that will allow you to listen to something so this could either be like your own playlist a Spotify playlist an artist or an album and so the goal that we were given is given a shelf order the media in order to maximize for total consumption time so basically we want to put the more popular items at the front of the shelf in order to maximize their total consumption streams okay so given this goal an ml solution to this problem could be the following so first we would train a classifier to predict the probability that a media item and shelf pair will be played and then given that probability we can order the media items in terms of that so based off this approach we have to make two decisions first off is model selection that is how are we going to even choose a model to model the probability of a stream given an input feature and then secondly the feature selection which is given a model what are the most informative input features and since this is a talk about feet ran I'm only going to be focusing on the feature selection components so that's basically asking like what is this X that you see all right so at the very bare minimum the number or the features that we're going to need at least are gonna be the shelf name and the card URI but or media of your I but we'd also like to add in some additional contextual information such as weekday time of day user location user attributes or a media item type and so on the right-hand side this is going to be our basis of our input features all right so we're gonna have our raw input features and then we want to go to our model input features so basically keep taking those either in categorical representations and converting it into some form X that can be passed into a model and so this is where feet Tran comes into play and so you can think of axis you know something like this basically a dense vector full of zeros and ones ok but before we started even using feet rent we first asked ourselves the question of should we even be doing this after going into our ml formulation kind of had an idea of what the model and the input features should be but we weren't sure that it was actually gonna work so what we initially did was we collected a small sub sample of user data from home where we had our raw inputs which from the other side where examples were like the shelf name in the media item URI and some additional contextual information as well as the raw labels which would be for example how long a user listen to a media item and so we initially experimented with model and feature selection in Python using Sai PI and SK learn and that allowed us to kind of really quickly and initially test a lot of different ideas but once we narrowed them down and we were confident that our approach would actually work then when we then we got to the phase where we said let's start actually building it and that leads to the next section entitled building stuff and so this is the end architecture that we created but let's first concentrate on just these three components the first being the common features Asian library which held our liked feature and aspect of it as well as common pre-processing steps and that's going to feed in both into a data pipeline and a shelf reordering service so let's first concentrate on the common feature ization library ok so the purpose of the common features Asian library is to take our raw data and convert it into an intermediate form that can then be extracted by feature on so to make this more concrete let's give this some an actual and example so our raw data is going to have information such as the time played the media item URI the shelf name and a timestamp and then from there our intermediate form is going to be called processed data which is also going to have the time played and then from the media item URI we're going to get to other features which is the media item ID which is which we use because for media items such as daily mix it's going to have a unique our eye for each individual user so it's not going to be particularly informative if we want to train a model in order to predict if you like daily mix so we're gonna first transform that across all users to a map to a common ID and then also the media items set type so this could be like a personalized playlist which is what daily mixes a Spotify playlist or your playlist will also have the shelf name and then from the timestamp we're gonna extract the weekday so the process data is really just meant to convey that we're gonna do data pre-processing and all this data pre-processing is going to take place in the common feature ization library and then the actual extraction step that's going to occur with either in the pipeline or the service so then the first case is we want to take that raw data and make it something that we can actually work with and so our common feature ization library is written in Scala so we're just going to make a raw data case class that holds these features and similarly with the intermediate representation we're just going to make another case class called process data that has all the features that happen after we do the pre-processing on the data and so the common features Asian library it's also going to hold two other functionalities one is the future spec that fowling talked about so this works out pretty well that the two transformers that I'm using in this example or binarize ER and one hot encoder so we're gonna create the label using the binarize ER and so we're gonna say if someone has listened to this song for more than 60 seconds let's set the label to one indicating that they they like this they enjoyed that experience and zero otherwise to create like- an example and then we're going to use a one hunting coding for the media item ID set type shelf name and and then this home feature Iser object that's going to contain the feature specification that we defined above as well as just the function that goes from the raw data to the processed data and again that's just meant to convey that all these pre-processing steps are taking place in this common feature ization library alright so the next step after we had this was building the data pipeline so let's take a look at that and Spotify we write data pipelines using shio which you some of you should know about it and so the two inputs to the data pipeline is one it's going to use that common feature ization library as well as like home logs so let's look at the steps that are going to take place in this data pipeline so first is simply just reading in the historical log data the second one is we're going to get the raw data and convert it to a process data and so the steps in this process would be one using the case class raw data that we defined in our common future ization library and just extracting all the fields that we need from our input data then calling the process data function again defined in the common features asian library and then finally we're gonna have a s collection of processed data and so the next step is just extracting the data using the home features asian specification and after that get the feature values and in our case we wanted it to be a sparse double so very only a few steps that you need to do before you can get the data in a that you want and then the final steps is just saving the training data as along with the feature in settings so in our case we wanted to convert the sparse vector to a format that could be easily read during our model training which was to just save it as a string and then also both save the training data of course and then finally save the fee Tran settings because in the end we're gonna want to you know recreate these vectors using the same process within the service okay so after the data pipeline is run we have two outputs one being the feature and settings and then also the training data and then so although I'm not talking about this the next steps would be actually doing the model training and then afterwards you're gonna have a trained model so this would be your H that you saw in the previous slides and you would get then the optimal weights associated with it and so finally that brings us to the shelf reordering service which is also going to be using this common feature ization library and the two inputs that it requires are the feet ran settings as well as the train model okay so let's take a look at the shelf reordering service so just a side note before I go into the details is that at Spotify our micro services are written in java using apollo which is an open source library and that we also generally use protobuf to send messages to the service so in our case we want the messages to have the media item URI along with the shelf name and then we can think of a home request as having just a bunch of this information and so that's going to be the input to the reordering service all right so let's take a closer look of the feature and components within this shelf reordering service so similarly as what was done in the data pipeline we have to get our data from somewhere the what we actually need to score in this case it's coming from the request and the protobuf and then finally doing a very similar functionality is that we get the raw data and we convert it to the processed data so again we're using this case class raw data that was defined in our common feature ization library the only important note here is that we're using a dummy label since in the data pipeline we were using historical logs so we had information about how long a user actually listening to a song whereas for here we want to removing going into the prediction phase I want to guess how long a user's going to listen to this song so for now we're just inserting a dummy label we're calling the same function again defined in the common feature is a ssin library called processed data and in the end we output a list of process data okay so then the next steps are is we're gonna get a java friendly version of the future spec I keep on saying that which is again defined in the common feature ization library we're gonna load the feet ran settings and again we get these feet ran settings after running the data pipeline and saving the feet ran settings and then calling the extract method except in this case for calling extract with settings and we're passing in the feet ran settings and finally we're gonna output our values using a sparse double so one important note is I mentioned that we're using a dummy label and so to then actually use the features for the purpose of prediction we're gonna strip out the entry that corresponds to a label and this basically involves just like a shift by one over it's just important that you remember to do it or else all your features are gonna be off by line so this is a little detail that I bring up you probably would imagine because we did it wrong the first time so just make a mental note for anyone else who plans on doing this in the future all right so what happens next after we run kietryn is we go from a sparse double to a another form that can be passed into a model get the model scores return the home content with the scores and then reorder home content using the scores so I want to now look at the shield data pipeline and the Apollo service side by side just so you can see how common the functionality of using feet Ranas in both of these settings one which is again geo data pipeline and an Apollo service written in Java okay so both initially they have to get their data from somewhere the sheet up she'll pipeline reads in historical logs whereas the Apollo service gets its message from like a user request that actually sends in the home content from there we're again using this raw data case class that we defined in our common future ization library calling the same method called process data again defined in the comic feature ization library and in the end we either get an S collection of the raw data or a list of the raw data in the Java service there are two additional steps that you have to take one is getting this feature and friendly version of the application and then also you have to get your feet ran settings which were saved during the CEO of data pipeline and then you're also calling extract in a very similar way the only difference is in the Apollo service you're calling extract with settings and you're passing in the feet ranch settings which balance showed you what they look like it's just a string that has all of the aggregations that were saved and then finally depending on the final form that you want to use you get your future values as either it could be like a dense array or a sparse array and then finally in just the shoe data pipeline the important part there is we always need to save the feet Tran settings and so again to really drive how similar this is let's go through this exercise one more time but have them swapped and also upside down but all right no I'm not actually going to do that it's getting late you're all here but the point I would make is if I if I did do that you might be able to follow along because feature and it's very easy to use within a service and also a pipeline ok so again let's just go through this architecture one more time so we started off with defining that common feature is a ssin library that made it really easy to use feet ran in both a Geo data pipeline and Java Apollo service from there we ran our data pipeline on historical home logs we got from that both the feature and settings and the training data from there we're able to run our model training algorithm and the output of that would be the Train model both the feet ran settings and the trade model are the inputs into the shell free organist and from there as like a user were able to query to get model scores okay so hopefully at this point you have known that or know that fee Tran is both pretty easy to use for both model training and within a service that evaluates the model and so what we did was we started with the problem and for that it was reordering content on home so that the more popular items would be placed in a more viewable position we then talked about how we could use machine learning to formulate the problem from there we decided to first validate the hypothesis by using a small sub sample of home data and initially prototyping in Python using SK learn in sci-fi and then once we were confident that this was a viable solution we moved on and basically developed the system architecture which had four main components being the common feature ization library the data pipeline the model training and the service alright so thank you for listening and questions [Applause] yes so it depends what we changed if we changed kind of if you change the feature specification and then we'd have to rerun at the data pipeline as well as the model training so every time or even if the data that we're using to train our model changes as well we have to run and everything again well thank you for listening [Applause] you [Music]