scala.bythebay.io: Shubha Nabar, Meta Data Science: When the world's data scientists are not enough
I you alright so this is a bit different from what might be on your handouts then I decided to change the topic based on the interests of the audience so I'm going to talk about this Scala based machine learning library that we built at Salesforce Einstein and for a bit of background about myself I currently lead data science efforts at Salesforce Einstein prior to that I was a data scientist at LinkedIn and a Microsoft and before that I got my PhD in computer science from Stanford University so I thought it would be useful for me to set up some context around all the work that we're doing at Salesforce as many of you know Salesforce has this highly customizable CRM platform that business is used to manage all kinds of customer interactions so for instance uber uses Salesforce to manage all its marketing automation and so this means that all the email marketing campaigns to its riders and drivers are managed through Salesforce comcast uses Salesforce to manage all this customer service cases square uses Salesforce to manage its entire sales process from lead to cash ucsf uses Salesforce to manage patient health care records so there's literally hundreds of thousands of businesses that are using Salesforce everyday to manage all kinds of customer touchpoints and the problem we're trying to solve is that for the majority of these businesses data science is out of reach and why is that well first of all to do data science you need to have data and so to get the data you need to have a team of data engineers What's Eating this data for you from all the relevant data sources and then you need to have a bunch of machine learning PhDs on staff in order to do predictive modeling on top of this data and then finally you need to have all the DevOps and infrastructure support to keep all of your processes up and running so it's extremely non-trivial and this is what we're trying to solve at Salesforce Einstein we're building a platform that will make it much easier for all these businesses all these hundreds of thousands of businesses to get predictive value from their data without necessarily having large data engineering and data science teams at their disposal and so that's that's the context of all the work that we're doing at Salesforce and in particular in the stock today I'm going to talk about Optimus Prime which is the modeling library that we use in Salesforce Einstein some of the characteristics of this library the reason we decided to call it Optimus Prime is because machine learning is all about transform transformations and some of the characteristics of Optimus Prime are that it provides a declarative and type-safe syntax for collaborative feature engineering and model development and it uses spark ml is the underlying execution engine before I before I dive into the details of Optimus Prime I wanted to walk you through a little bit of the history of how we got Devere where we are today so two years ago we were developing the v1 of this library and our design considerations at that time word that we wanted to build this on top of spark and ml lib so that we could use all the nice in-memory iterative computing capabilities that spark provides at the time however spark ml was still in its infancy and in particular the data frames and mo biplanes API were still being developed and another interesting thing about our use case was that data preparation and our use case involved repeated aggregations over massive amounts of event data and so we needed to have a way of doing this really efficiently so what we ended up doing was we centrally built our own ml pipeline interface that operated on our dd's and we developed our own data transformer interface and on our own execution engine in order to optimize how these repeated aggregations were executed now what happened in the meantime was that spark ml just kept getting better and better because there was an active community of contributors that was making it more and more awesome and when we looked at their api's we felt that they'd really nailed these really had the right set of abstractions everything was either a transformer an estimator or a bike line and we wanted to be able to leverage all of us awesome enos but at the same time we felt that it was still less than ideal for our use case which was large scale collaborative feature engineering and model development and in particular in the Salesforce use case it's not going forward it's stuck oh there we go in particular in the Salesforce use case we are dealing with lots of very structured data with a very rich schema and we have many different teams and so what this means is that we want to do lots of really rich feature engineering on top of this data and at the same time we want to do this feature engineering in a reusable way because we have many different themes that are all developing a variety of different predictive apps on top of these common data sources and so we don't want to have 20 different themes developing 20 different implementations of tf-idf each of which are a little bit buggy so we want a lot of reusability and then we also want another interesting thing about our use case is that the developers who are using this platform do not necessarily have deep spark ml expertise right so what we would this with this use case requires is that we provide a very simple declarative typesafe syntax but makes a machine learning development easy in the company and so this is why we built optimus prime V 2 it's a developer friendly interface for non-expert users and this time we were careful to not reinvent the wheel so we use to spark ml as the underlying execution engine so now I'm finally going to get to some code and Vitaly my boss asked me to apologize for only getting to the coded line at slide 13 but i think it was useful to set up some context here and again the slide is not going forward why is that ok the exception yeah all right and then go to that slide and start the next one alright so apologies for that so so all of the goat examples that i'm going to show now are going to use the titanic data set it's a very morbid data set that's often used in machine learning examples it's a it's a data set that is all the passengers who are on board the Titanic and often the machine learning use cases to predict survival and some of the features that are in this data set this is a sample of the features are the passenger class the passengers age and gender the number of siblings and spouses they had on board the number of parents and children they had on board the passenger fare and so forth so let's take a look at the new syntax so first in order to you know show the benefits of the new new syntax let's look at how you would define features in spark ml so if I wanted to create a new feature in spark ml that's based on these existing features in the data set and this say this new feature I want to create as the family size of the passenger on board so I want to sum up the number of siblings that the passenger had on board and the number of parents that the passenger had on board this is how I would do it so when spark and mal i would first create can you see that font ok and spark Amell i would first create a function that takes two integers and Sam Sam up I would then register this function as a UDF and then I would transform my training data frame by applying this UDF to these columns to the siblings column and the parents column and notice here that I have to be I am referencing these two columns by their names so to be very careful that I don't make any spelling mistakes here that I know exactly what the names of these columns are I also to be careful that these are actually integer columns because if they're not I'm going to get a runtime error and then I have to also keep track of all the output column names that I'm generating and then the other interesting thing here is I'm actually mutating the underlying training data frame so if I want to apply the same transformation to my desk data I have to be very careful to copy over the same code and not make any spelling mistakes so it's very error-prone and a lot of fares are only at runtime after your pipeline has been running for an hour so in contrast an Optimus Prime this is the syntax that you would use for defining a new feature you would say that the family size is a numeric feature that is obtained by applying that function to data of type passenger and so there's two interesting things over here we've introduced the notion of a feature which is essentially a type-safe pointer to a column in the data frame and that's what gives us all this type safety and the other interesting thing here is that it's not mutating the underlying data it only depends on the type of the underlying data so you could apply the same feature definition to any data of type passenger and there's a very clean separation between feature definitions and data now let's take a look at how you would do feature transformations so going back to the spark ml world if I wanted to create a new feature which was the normalized family size this is how I would do it I would create a new normalizer I would set the input column to be my family size column and said the output column to be a new one I would then fit this normalizer to my training data and then I would transform the training data and again I have to be very careful but I haven't misspelled anything here that I correctly copied over the name of the previous output column and so forth so in contrast an Optimus Prime this is what you would do and I think I'm stuck again oh man okay Wow yeah let's go to the next one alright so in contrast and Optimus Prime I would create a new normalizer but where I said the input column to be the family size column but the nice thing here now is that I have a vowel the family size is a vowel there's no chance I'm going to make a mistake when I'm referencing it the other nice thing is that because family size is this strongly typed numeric feature i can create these nice set of transformer shortcuts and so i can shorten this do family size normalize and similarly if I had a feature such as the age another numeric feature I could have another feature that was the age times the family size that I would define with this very nice simple intuitive declarative syntax and so so the type safety has sorry go on yes yes are we saving the min/max somewhere but at this stage is all definition still by the way we haven't run the pipeline when we actually run a pipeline and save it that's when we save all that information um oh the question was do we save the min/max and everything that's needed for normalizing alright so so type safety has really provided you know it's a reduction in runtime errors and it's also provided this really nice intuitive declarative syntax and and the other nice thing is you get compiled that you can as you're editing your code you can get code completion in your editor because with Scala compiler kicks in but what would what the type safety has really provided for us is transparency around what are the inputs and outputs that are expected at every stage with your data transformation pipeline so for instance let's go back to spark ml say I wanted to create a new logistic regression model and fit it in my data this is how I would do it I would say I want to make a new logistic regression estimator fit it to my data frame and dig that fitted model and use it to transform my data frame now does anyone have any idea of what inputs this this estimator expects any guesses so if you dig through the documentation you'll find that it expects that the underlying data frame has two columns one call features and the other called label does anyone have any idea what outputs is going to produce close so if you run it and you you know you materialize your data frame and you look at your outputs you'll see that it produces three new columns vector vector numeric call probability Rob prediction and prediction so it's it's you know this kind of thing can lead to a lot of tribal knowledge so it's needed and it's not very easy to get started and so in contrast and Optimus Prime thanks to the type safety and I'm stuck again wow this is sorry yeah so in contrast an optimist friend this is how I would I would I would define the logistic regression estimator I would said exactly the input columns that I want and thanks to thanks to the type safety I know exactly what output columns to expect a couple of 33 features vector vector numeric and and so really the vison deck says it has provided us with four main things first a very clean separation between the feature definitions and the data second type safety which means a reduction in the number of runtime errors and also very nice things to just go to completion while they are actually writing your code so a lot of improved developer productivity third it has of the the third thing that it's provided is this really nice syntax where you can have shortcuts for your transformers and finally and perhaps most importantly it's provided transparency around what are the inputs and outputs that are expected at every stage of your data transformation pipeline but Sir then how does this all actually run as I mentioned before we use spark ml as our underlying execution engine and so all of our optimus prime did it just go upside down wow it is going on and now it's like gone well I have no idea what's going on all right so the other so so what happens is under the hood all our optimus prime pipelines actually translate the spark ml pipelines so spark Amell has essentially these two fundamental concepts of transformers and estimators a transformer is something that takes a bunch of data frame columns and produces a new column and in this estimator is an algorithm that's applied to a bunch of columns to produce a new transformer and so a spark ml pipeline is essentially a bunch of these transformers and estimators stitched together and our optimus prime pipelines are essentially a wrapper around all of this when you use our syntax to declare your optimus prime pipeline and then actually execute it what happens is under the hood it automatically creates the dag of all the spark and mal transformers and estimators that need to run creates a spark Amell pipeline and then runs that so we've left it a spark Amell to optimize the execution of these pipelines and what would this is really enabled with the way in which this is made the way in which this is transformed development at Salesforce is that it has enabled much more collaborative feature engineering and model development and this is because because of the clean separation wow this is strange No if it doesn't invert it's weird um okay yeah you can also just be okay yeah or you could save it as a PDF right now that might help okay um [Music] oh sorry let me yeah let's just finish it off yeah so for instance because of this nice separation that we have between the feature definitions and the underlying data we can create this object of our Titanic features where we just list all the features that we want to use and and so now jeez and so if I want to now use those features some of those features to create a new model I would just import the Titanic features and a new model is basically just good no it did it did I said the input features explicitly to be survived and I don't want to go back cuz I don't want this so the new model a new model would basically be just yet another feature transformer that I can contribute back to this repository and people can use it without having to understand the details of how this model was built and the other thing so so basically this is bootstrapped the development of new models which is very useful in our environment where we have many different developers building many different apps on top of these common data sources it's also tremendously improved developer productivity because of the run time because of the compile time checks and all of this was possible because if type safety so if you want to find out more about how it's not going forward if you want to find out more about how we actually built all of this you need to stay till the end of the day and someone from our team matthew talvin is going to be talking about how we actually built built this so and i really wanted to go to the next slide so i'm going to exit again Wow okay nevermind so what I want to say is that we're okay okay so what I wanted to say is that this is just a small piece of all the work that's going on at Salesforce Einstein and so if you're interested in knowing how prediction I oh and meta mind and heroku everything fit together under the Salesforce Einstein umbrella please email us reach out to us at Einstein recruiting at salesforce com we're way over time yeah go ahead we're planning to it's not yet yeah is any of this open sourced not yet but we we would like to eventually you