scale.bythebay.io: Chris Rupley & Till Bergmann, Complex Machine Learning Pipelines Made Easy
thanks for coming out appreciate you coming to join us my name is Chris my colleague till and I are from Salesforce we're gonna talk to you about some work we've been doing on machine learning pipelines and implementing them for a few different applications we're working on at Salesforce so you know what we're gonna be getting into today is what kind of data we're dealing with the Salesforce to start with and then what kind of machine learning we're doing on that data and then just a brief overview of what we talk what we mean when we talk about pipelines and then the book of the talk is till and I are going to be talking about a few different use cases for these pipelines and things that we're doing with them one more specific use case that has specially tuned pipelines and one more generic case that has much more flexible pipelines so first of all the data at salesforce salesforce is sort of broken up into different business units we called clouds they're broken up by function so they're there the first cloud at salesforce with the sales cloud so it has data related to different sales processes sales leads accounts contacts opportunities there's a service cloud that deals with service cases and there's a marketing cloud that deals with marketing email campaigns those sorts of things and there's also a custom type cloud custom object so salesforce is also a developer platform so outside third-party developers can build applications that work with salesforce data and store data and sales floors and they can develop their own objects and their own schema to store this data and you know there's many other different clouds within Salesforce IOT analytics those sorts of things but the challenge here is all these all these different clouds all these different units have different data different schema different organization so we're gonna need to figure out a way to get these into our pipeline and deal with all of them the advantage that we do have though is that our data has a rich schema and metadata associated with it so we're storing our data in a database it's not just a text field it's an address field it's an email field so we know more information about it so that allows us to be smart with how we handle it in our pipelines so we don't have to do a lot of that processing on the front end the other nice part is we have we know the relationships between the data as well so contacts are linked to accounts which are linked to service cases and we can do more with that from that perspective as well also it sails first we have the notion of multi-tenancy so salesforce is a business to business company we deal with hundreds of thousands of different companies they all store their data in salesforce but the issue is you know every customer wants its own data separate right they don't want their their data to be going to benefit another customer potentially a competitor so we silo every customers data separately so all the different customers are going to have self-contained data and and the other thing you know about the different customers is they can vary considerably both in size and scope of the data and the dimensionality so you might have a non-profit like Kiva which has a very very different business process than someone like Comcast or asana so there's there's large differences in the the amount the shape the scale of the data that we're dealing with and like I said we only use a customer's data for their own benefit so if we're building machine learning model we're building it with that customers data to make predictions on that customers data we often use the term medium data at scale rather than big data because each all of our data is partitioned not only in a different clouds at salesforce but different customers so that we're working with small bits of the data at a time rather than dealing with a big big amount of data all at once so how do we do machine learning with this kind of data we're developing an internal package we call Optimus Prime which among other things allows us to build pipelines quickly better flexible machine learning pipelines and we can think of it as an improved interface for spark data frames so Optimus Prime is an interface to spark which of course is on top of Scala and our package is of course built in Scala as well so before we get into things too far let's do a little background on what I mean when I say machine learning pipeline so typical machine learning pipeline is going to look something like this you've got your raw data that you know you probably store in a database and in some kind of structured format you're gonna do an ETL step extract transform load so you get the data out of your database into some form you can use and get it into memory in this case the the way we're representing it it's gonna be in a spark data frame it's a vectorization step so we take all that data that's now in our data frame and convert it into some numerical form because we're gonna take we're gonna take this data and perform some operations with it and it needs to be in a numerical form the reason needs to be numerical is because we're putting it into some statistical modeling techniques so if we want to do a classification regression recommendations we need that data in the right form and so with those statistical models then we're gonna use them to produce scores or predictions on any number of things if it's a classification for example we'll build a person unsubscribe or not if it's a regression you know maybe the expected sale price of your product recommendations as well we can predict the most likely to buy or the highest rated item for a particular user so let's get into the use cases the first one that I'm going to be talking about is called case classification this is a special application of these pipelines so what I mean when I say case classification so this is related to this the service cloud part of Salesforce where from a customer service perspective someone will request some kind of service it'll open a service case so if you're having having a problem with your internet for example and you email your Enter provider and say hey I'm having a problem please fix this that can come into if they use Salesforce that will come into the Salesforce platform and now that can be a lot of work to triage all those emails to you know decide is this email important I need to address it right away can I wait what's the issue who does this problem need to be assigned to so that they can fix it so that you can do it quickly and effectively so the problem we want to solve here is you know given a whole bunch of history of someone's service cases can we now predict when new cases come in what the priority is what category they're going to be and beyond that for each individual customer that's that's using this product they may have custom things they want to predict things that are specific to their business process that exist in their Salesforce data that they may might want us to make predictions on as well so we want to be able to take in you know any number of different labels different text and classify it automatically for them so what that looks like is something like this so say you've got an email that comes in you know I'm having having a problem with my shipment and it's going to have different tags on it for example so this one is classified as medium priority it was a issue related to shipping within that issue of shipping the specific problem was an address correction so you know a customer will have a history of all this data and this is what we're starting with so you know there may be any number of issues different priorities different categories and so on so these emails come in someone has triage them in the past they've stored this data so the problem is going to be a multi-class classification problem it can take any number of different priorities and it's also going to be multi field classification because we're trying to predict these many different fields a priority the category the sub category in this example so to do this we need a more flexible data pipeline one that can handle all the different customers all their different data and handle all the different fields that we want to predict in this case so what that looks like you your data coming in on the left we're gonna extract out that email field we'll vectorize it into a numerical form that we're going to call our features then you're also going to pull out the field you're trying to predict the priority field in this case vectorize it as well we're gonna call that the label those both go into our statistical model and out of the statistical model comes our predictions now we also need to do this for the other fields the category field and so on and so on as many fields as we need to predict so so this is the the overview of the pipeline we're trying to create here now each customer is going to need a different pipeline obviously so you know say asana wants us to predict with the priority the category in the subcategory are going to be from emails Kiva might want totally different fields Comcast has a lot of stuff they need to predict and so we want to be able to scale our pipeline up and down to be able to Train all these models and make the predictions that are requested for for any customer thus looking for them so what does that look like what how do we generate this pipeline so here's inside the feature of vectorization that we're working with so you get an email come in that comes in it's a bunch of text so in spark if we want to deal with that we can load up a data frame so you load up that data into a data frame but we take a different approach with our package of salesforce rather than dealing with data frame directly we think in terms of the features that we're creating so instead of loading the entire data frame we're gonna extract that email feature out and we have a notion of a feature builder that does some of the interesting parts of it here first of all like I said it's a feature another data frame type that we're dealing with and we have a static statically typed feature so we're extracting a feature of type text and we also have statically typed sources this is coming from a case object so because we know those things we know a considerable amount about the structure the schema the sorts of things and different transformations that we can do with them so the next step in our pipeline we want to tokenize that data so you split up that blob of email text into different words you can do that in spark there's a notion of a tokenizer the tokenizer does that you set your input column and your output column and a data frame and you're good to go handle it the way that looks and Optimus Prime we've got an email tokenize method building to our features so our input and output features are implicit here so we don't need to specify those specifically and because we know the types of these we can do a type 2 transformation so that email is a text field and we know a lot of times on text fields you're gonna want to tokenize them so they have a tokenized method as well so this you know the same the same things are happening on the back end but we have a simpler interface for them so there are a few other things you might want to do in your pipeline after you tokenize it you might want to remove the stop words so those are common common English words it might just add noise to your model and spark has a stop words removal so again stop words remover so again you can you can specify it like this us at an input column and an output column something else you might want to do is limit ization where you reduce words to a common root so for example having have had will be replaced by the same root word again to reduce noise and the many different versions of the same word there's no there's no version of this there's no limit ization package and spark but you can do this from an external package if you import this but but we can accomplish all of this in a single step and Optimus Prime agains because it's a common transformation one way to do this is to in our tokenize method you can specify a default language so if we know that it's English we can remove the common English words we can do the common English limit ization and then the the last step that we want to do in our feature pipeline is to actually create you know create a numerical vector to go into our statistical model so in spark there's a sashing transformer which hashes eastward into a bin you set the number of features said you know as you said a couple parameters on it and again input and output column and then the way that would look like an Optimus Prime again the first step is to do the limit ization and then we have a vectorized method that applies to that so after its lemma ties that returns a feature of type text list so it's a list of a list of strings basically and because we know it's a list of strings we can we can vectorize that as well and we've got the same you know the same thing going on in the backend the hashing transformer so we specify the same parameters but again it's more concise and type safe syntax and if you want to use a different vectorization method for example TF term frequency tf-idf those are available as well so let's get into the modeling step next so the way a model works in SPARC you'll give it features in a label and it will return a prediction a probability and a raw probability column this is the what will eventually turn into our predictions and that can be specified in SPARC like this you have a logistic regression model you set the input and output columns and you can set your parameters for that model one thing that's not specified in here is where your outputs go so it's it's inherent in each different model what outputs come out so logistic regression will produce these three columns for your data frame with these names and that's an implied and it's not explicitly controlled here and Optimus Prime this is what it looks like so we take that feature vector we created this will now have a logistic regression method we can apply to it and then we specify the label column associated with it and we specify the parameter so we're getting specifying the same things except now our outputs are going to be these three new features of prediction the raw probability and the probability now say we want to take that a step further instead of training a single model it's typical that you want to do cross-validation and train models with an array of different parameters you can do that in SPARC there's a pram grid cross-validation you set the array of parameters that you want to use and it will iterate through them and choose the best one based on a certain evaluation method an Optimus Prime we also have a built in method for doing that it's the same logistic regression method except now we can specify specify our regularization parameter instead of a single value and it will do the exact same thing on the back end so a step beyond that say we want to Train multiple statistical models in SPARC so not only oh just take regression we're gonna try a random forest decision tree naive Bayes model in SPARC that becomes really challenging each of those algorithms has different different outputs and they don't always blend together and it's it's not so easy to do that kind of model selection but we have built that in top news prime so we have a generic classify method that will test all available models for classification in the package with default parameters and a much more concise syntax and lastly say we want to combine all this together we want to train multiple models we want to do different parameters for each model and it you know a huge parameter grid and we want to pick the the absolute best model for a particular customer for their particular data that gets really really hard in spark so to do that in Optimus Prime it gets long but we can specify again a sequence of all the different parameters we want to try for each model within this classify method and what it will do it'll either iterate through each model through each combination of parameters and pick the very best one in performance in terms of their cross-validation so so what does that whole pipeline look like in code if we want to make it flexible so if we want to predict many different fields we can start with the sequence of this fields we want to predict we're going to want to predict a priority a category a subcategory field for this particular tenant so next step is to create a sequence of features that's just going to be a simple map operation we're gonna apply that feature builder to each of the different features and this extract them all out so now we have a sequence of labels of features and the next step with those labels is we want to classify them we want to train a different machine learning model on each each set of labels and features so again we can apply another map operation and and do a classification for each of those different fields so now we can we can specify which models we want to Train what parameters we want to try for each field for each customer will pick the absolute best one so again we see that scalable pipeline it's as simple as passing in a different sequence of fields to predict for each customer so so what are the things we have done to make this easier and Optimus Prime we treat features as a first-class citizen so the developer is working directly with the features not the data frames the dependencies inputs and outputs are handled automatically and it's gonna be much simpler to iterate we use a functional syntax now so that makes it reasonable maintainable although you know all the benefits you get of using a functional syntax you get here we can do model selection over a much broader range of models much broader range of parameters so that allows us to to dial in to to the most accurate model we can get for the particular situation and lastly we have type safety so you know in SPARC you're dealing with with columns and with the string name of the column when you want to extract them but with type safe features now we're able to catch issues at compile time rather than runtime in the middle of a giant job you can get you know the other bonuses like code completion in your IDE and those sorts of things that make development a lot a lot faster and more accurate so up next I'm gonna hand things over to till who's gonna talk to you about a more general pipeline that we've been alright yeah hey everyone so chris has shown you how to canal a use Optimus Prime to build a pipeline for one specific use case which will work great for that use case but obviously not for any other use cases and that's the whole point of it but as you remember there's a lot of different use cases it's done of doing different clouds sort of different schemas and the problem is because they can also be custom like every customer in Salesforce can create their own custom objects and they do a lot of them and we just simply there's not enough data scientists at Salesforce or anywhere already that we can go in and for every single schema can have one of those specific pipelines there's just not enough time and you know for our data scientists and you know this is pretty obvious a lot of these problems these common use cases of what customers want to do boil down to two different things the other classifications for example you wanna you know analyze churn and attrition or it's some kind of conversion from one status to a different status or you know the other big pad is regression but you have a number field and you want to predict this number for example the revenue in the next year the number of days until something happens so for example until the bill is actually being paid so you can how you know how much money will you have at the end of the month and I can show you quickly hopefully because this is a live demo how a customer right now in Saul's boss would actually solve this problem for themselves if they want to build the classification so so here you can see this is actually how Saul sauce looks like you have a bunch of different objects these are the different rows these are called contacts and it's say for each of those contacts you want to analyze well how likely is it that this customer at Ritz and it stops being a customer in some timeframe so all you have to do right now it's just use design same prediction building you make a new prediction you give it some name so just cool demo you choose the object that you want to predict so there's a lot of objects to be in Salesforce so you use the contact object here for this use case you simply select the field for which you want the prediction here we want to predict if someone at rit's so does it feel called iterative which is just a formula basically of you know true and false and you can then use all your fields that you have in all the data fields that you have in there you can use that to actually build your prediction I use it as features usually this can only feel this around 500 fields or something it's a lot longer and obviously you have to some way you want somewhere to store this prediction so you can just choose the field where the probability then gets fed back and this is all that you have to do to kind of like create this custom prediction for your use case so let me show you how that actually you know what actually happens when we build this prediction so each of this after this UI or like what we kind of need to build this pipe a pipeline for these custom generic use cases is we need to know about which field to predict which feels to use as features and what the data types are and as you've seen in the UI we already get that from the UI and the types we get from the rich metadata that Chris was showing earlier we know whether it's an email field and we know whether it's a picklist but that's a string and that kind of stuff so this is just simply how we store this con it's very simple we just have a response where we have the field and the type in this case it's a treated as the field type its boolean and then we have a list of fields that we want to use as two predictors so here's a couple of examples and this would usually go on for like a couple of lines and this is all you need to know to build these predictions for any kind of use case and so Chris really went into Connecticut deep look into like how we build these transformers and how we how we can elect transform the features and that's possible because he knows well he's gonna work on the case object he knows all these fields exist we don't really know what type what what actually these fields have what kind of information they store we only know one thing and that is that type so for example when we know it is a there's an email type so we might want to extract the domains so you know what maybe what kind of company do you belong to what they add sign is right and then you want to pivot and keep the top K maybe the top ten the top twenty common occurrences of that domain and then we need to also vectorize it so we need to do all these single steps but you know if you have a pick list the categorical value you don't want to extract anything you just want to keep the top k and then also vectorize it if you have a phone you might want to check well it's just a valid phone number or is it invalid what what's the area code all of these things you can do and then again you need to put it into a vector if it's numeric you can do multiple things you can do you can normalize it put it into a vector you can bin it and then put it into a vector and this goes on for all of the different cells for types that we have which there aren't like twenty or thirty types so if you want to do this for every single field that we get from these customers and it is hundreds of fields we would probably never finish writing the code because it's just too much work for a company like Salesforce where we have unknown kinetic data entrance entry practices from the from the customer they don't tell us how to actually collect this data one really useful thing for us is also for all of these features we want an indicator whether that field was null or not and the magic of kinetic optimus prime here is that we can simply just call one faction to do all of this and that is just called dark vectorize on it and it does all of the things that I described for all of the different fields the problem with this is that now we have not only like hundreds of fields the hundreds of fields magically turn into hundreds and hundreds of features because if you pivot for example in to taunt tap top-ten domains you certainly don't have like one column you have ten columns and there's multiple problems with this well a lot of these might just provide you with no information gained in the model it's not gonna harm your model but it might slow down your model building a lot so that might be really low correlation between your feature and the outcome the field might actually be the same for all rows so it doesn't carry any information and you want to throw that out as soon as possible because it just slows down your model building another problem which is very bigger is if you remember this feature list that I showed you is a field in here called a treated date now we're trying to predict if someone is a treated or not that field at rooted date it's not gonna be filled out if that person hasn't treated yet which is a huge problem because our model you know math is even though it's math it's kind of dumb and it'll just you know there's a perfect correlation and it will just use that field to predict whether someone is a treated or not which is not really helpful because maybe give the predictions that field is always going to be null so this is called label leakage and we definitely want to catch that because that is actually harming a model our model will not really perform well so this is again this is a lot of work if your data scientists you spend hours and hours trying to catch all of these label leakage things you try to catch already optimized all of your features using Optimus Prime you just have to call one function again which is dot sanity check which is what we call this sanity checking and feature selection and to quickly show you what the whole pipeline actually looks like for this generic problem it's really only three functions all that you have to do is called that vectorize on a sequence of raw features which will automatically pick the best transformer creates the best features for your field type you have to sanity check it make just simply one function again and then you call classify if it's a classification problem and if it's a regression problem you just switch it out or classify for regression so just to kind of like iterate or just to summarize again Yop makes Optimus Prime makes these complex pipelines really accessible easy it's Chris has shown that for streamline piping creation for one specific use case you can use it to build these really in-depth pipelines really well and easily wave easier than in spec but also my which is really helpful for developing you have this Auto and malfunctions which you can use for these flexible pipelines so for any use case that you really want to throw it at it's really easy to get started and develop pretty good models so you have smart feature transformations based on types you have this I need to check and the featured selection which gets rid of unnecessary and harmful features and you have the easy model selection which will just do cross validation and run different kinds of algorithms on your data without you having to specify any of them alright to summarize it makes complex machine learning pipelines easy and that was it thank you yeah I think we have time for questions so for the sanity check I assume the data leakage check is just too high correlation between context and fuel to try and predict how do you do like the not so useful feature printing so there's this they're not so useful things there's multiple things that we do so we also check for low correlation so not so you know if the correlation is very close to zero you can also remove it relatively safely the other thing is also that in a model selection usually is pretty high regularization in the in the logistic regression or something which will so you just have the feature in there it'll just won't be used in the model and also like I mentioned low variance right if it's the same if it's the same feat if it's the same value for all fields and the variance is really low it also gets removed yes for all rows right if you have hundred rows and you know the country is the same for all of them then it doesn't really carry any information so I want to refer to revelation problems they also monitoring of liars and noise for example so we get rid of layers from that so they not to scoop the result so the yes so it's a good question with outliers so we try to get rid of them in what we try to catch them in the sanity check already right with like correlation we also do I'm for categorical features we also do cry must be to kind of like yeah see what the correlation is between the features I think if we do regression we don't nothing happens if there are outliers as such but we usually try when we actually you know look these two sometimes look at the models it's not completely hands-off but pretty hands-off you know we try to see why if there are less why those a cost and usually the way we do it is you know we look at them and usually the outliers happen because there's some kind of data entry error or something and then we can kind of like feed that feed that back to the customer to how did you talk about the name Optimus Prime I don't know how we came up with that No so it's because it's essentially a series of transformers yes we might have to change the name for legal reasons at some point so yeah watch out for different for a different name soon so I understand you're gonna possibly open source that you don't throw a pilot for this so learn the barriers to that the name no sir I think we are currently in the process of kind of looking for outside customers who will be who are interested in using Optimus Prime I if you are interested or anyone is interested I can refer you to the right person to talk to which is not me and yeah it will be the plan definitely is for this to be open sourced at some point but I don't know what the timeline is but I as I said if you're interested come and talk to me and I can point you to the right person to talk to you know yeah no questions [Applause]