Scale By The Bay 2020: Antje Barth, Put Your Machine Learning on Autopilot

Scale By The Bay 2020: Antje Barth, Put Your Machine Learning on Autopilot

Recording: Scale By The Bay 2020: Antje Barth, Put Your Machine Learning on Autopilot

[Music] so thanks everyone as well for joining this session so as devin said my name is anjia i work as a developer at wikit for ai and machine learning at aws and i want to show you in the next 30 minutes how you can put your machine learning on autopilot so let's have a look and just a quick note up front i'm also co-authoring a book right now an upcoming o'reilly book data science on aws together with a colleague of mine chris frackley which a couple of you might know so he's also giving a talk today at the conference about multi-unbanded models so super exciting check that out all right so for this talk what i'm going to do is i'm going to quickly introduce the traditional machine learning workflow to kind of show you how much work it actually is to build a model from scratch to prepare the data to train the model and i'm going to use a pre-trained bird model so we're going to see an nlp use case and all of the work that goes into fine-tuning such a model and then in the second part i'm going to show you how automated machine learning can actually help in automating a couple of those steps and really make our life a little bit easier and both parts come with a demo so we're not seeing too many slides here and also see how machine learning and automated machine learning work in action all right so let's get started so i assume some of you already work with machine learning projects so you should be familiar with that workflow before you even start developing your model you have to do a lot of data preparation right so you're getting some data and you're going to analyze the data you're going to check if the quality is okay if there's any missing values etc and then you need to transform the data right to feature engineering to fit the data to the machine learning algorithm you're about to use to maybe combine a couple of data columns to features etc you validate that the data is correct and then you create a training data set once you've done the data preparation you can then start with a model development so you develop some training code you pick a couple of algorithms maybe you want to do some classification so you're going to choose maybe an extru boost algorithm and then you're starting to to prototype right you develop the code you test the model you validate the performance the accuracy and keep iterating right over and over until you find a really good performing combination of model parameters algorithm together with your training data and once you have that you can then what i call train at scale and this is where you really want to use all of your training data available and you probably want to implement some distributed training to speed up the process etc and then once you have your trained model you can then deploy it into a production environment and start serving predictions to your applications and even then you'd still have to keep monitoring your models and lock the outputs to see if there's any drift happening if your model may be decreases in performance with a live data and this is pretty much always the case you want to have a really close look at how the models are performing and then when new data arrives maybe start a retraining and create a new model all right so a lot of work for the machine engineers and data scientists now let's have a quick look into the data prep and the model prototyping phase because those are really the steps where automated machine learning can help you with so looking at this first stage around the data preparation i briefly mentioned it so first of all you're going to analyze your data right so you calculate statistics such as missing entries calculate quantiles check if the data is skewed in any way and also check the correlation of the data with the target attribute you want to predict right and you might even dismiss a couple of data points columns features because they are not correlated with the job you want to do with predicting a specific target you also need to define the problem right is it a regression problem is it a classification problem what is it what you want to do with this data set and then you're choosing the algorithm right for example for classification extra boost is a very popular algorithm but there's many more so depending on the problem and the data you have to pick and choose an algorithm or a few algorithms that's the different performance across and then keep iterating and also for the data set itself you need to kind of identify the schema right type of the content is it numeric categorical is it natural language right if you process data it's um text data it's completely different from processing numeric values so and we're going to see this as we're fine tuning our word model here in a little bit and then finally you have to transform the data also known as doing the feature engineering where you're really kind of transforming the data into the inputs that are optimized for your algorithm so for example for the natural language model we're gonna ingest raw text but we need to transform this in either word vectors or embeddings which the model can understand all right so after the data prep we then prototype our model and i kind of split this into two phases as you can see here on the top so the first phase is really the prototyping and this is where you experiment right normally within a notebook environment you just have a maybe a smaller subset of your data and you choose a set of hyper parameters which are the model parameters and you evaluate the model keep iterating change parameters see if the model performance is improving until you really have a satisfactory model and then the next step is to productionize or productize the model right this is where you really train at scale and you could leverage gpu instances gpu environments to accelerate the training over a large data set and then really kind of burn the model and make it ready to be deployed into the production environment all right so this was a little bit of introduction about the traditional normal manual machine learning workflow and let's actually see how this looks like so i'm going to jump here into an aws environment and for those of you working with machine learning this environment should look familiar this is a jupiter notebook environment and this is really traditionally where machine learning engineers data scientists start prototyping developing models it could be a local ide or it could be a hosted jupiter environment in the cloud which i'm using right now here on aws right so i said we're starting with a manual process so in this case first of all i need a data set and i'm using the amazon customer reviews data set in this example which is a public data set you can use for research and it consists of over 130 million of customer reviews coming from the amazon.com marketplace so whenever you go on amazon.com and you look for a product you might always check the reviews right if other customers rated it with a five-star really great product or maybe a one-star rating which is well i wasn't really satisfied with this product so this data set we can use and actually um use for nlp use cases it's really really nice and this is exactly what i'm doing here so we first of all need to analyze kind of the the schema so we can look that up so we do have a marketplace information in here customer id which is a random identifier so don't be scared a review id product id etc and what we're really interested in is the actual star rating and the review text which is called the review body here so we're going to have a collection of customer reviews for products and what i want to do is i want to build a model which classifies the reviews into those star ratings from one to five with five being the highest all right so first of all when i'm on machine learning engineer data scientists i'm gonna analyze and visualize my data right so here's a lot of code you probably write some visualization code as well and you're checking how the data looks like right to better understand your data set so in this case i'm running a couple of queries here for example calculating average ratings per product category and here you can see by the way the highest rated category by average is gift cards so with a 4.73 of a star rating interesting and maybe you're asking what's the lowest rated and we can also have a look here it's digital software actually all right so this is just a little bit of the first step and then you want to maybe analyze your data right a little bit more in detail so you could leverage a couple of tools i'm using a tool called dq here dq lets you run data quality checks over data theft and here i'm using it to for example check if the review id is complete in all of my columns here of all of my rows in the data set i'm also checking any correlation between total votes and star rating maybe i'm calculating the mean of star rating etc so you're going to do some checks you want to perform on your data set and make sure this is in good shape all right let's assume our data is is okay then the next step is to do feature transformation feature engineering and for our model here what i'm going to do is i'm going to use the ingest data consisting of a star rating which is my label and a review text and this is going to be my raw input training data and i need to convert the raw text into word embeddings that my pre-trained bird model expects and bird is a very popular architecture in the space of natural language processing if you're not familiar with i'll gonna also give you the link to this code repo where you'll find all the notebooks and here on top there's a couple of more links to slides and videos which explain the bird architecture in more detail all right so the feature engineering is to take this raw input data and then convert the text basically into those burden bettings all right i'm doing all of that i'm setting up my environment here reading in the data adding maybe a little bit more data to to see how the model is performing here and the next step really the big step is to write a processing script right so i developed my pre-processing script here's a simple python script which actually holds in this pre-trained bird model from a library called hugging face and it's also coming with a tokenizer and the tokenizer is used to basically split your input raw text into tokens and then we're going to encode those tokens into those embeddings so this is really everything you would do as the machine learning engineer here a data scientist write all of that code and then run the job and basically process pre-process your data once that is done you can then start training the model and what i'm doing here i'm not training the model from scratch i'm in a lucky position that in the nlp space there's a lot of pre-trained model so i'm using a pre-trained model from the hugging face library and this is already trained on a large corpus of books etc so what i need to do is really just fine tune that to my specific data set you can think of it like as transfer learning if that drinks a well so i'm kind of taking the pre-trained big model and really just adjust it to my specific data set which are those customer reviews so in this step again i'm doing a lot of steps here i'm checking where my pre-processed data lives i'm also probably want to set up some experiment tracking right so i'm pulling in here a tool that aws offers to track my experiments at the preparation and the training step here and then just going scrolling widely here um tune it for the specific environment set up for example distribution strategies how to read in the data set hyper parameters and then at one point i have my training script so i'm developing the code to actually run the model training all right and then at the next step i probably also want to tune the model find the right hyper parameters and hyper parameters are kind of the model settings and i i need techniques to kind of help me automating this as well right i could do it manually but normally i guess many leverage tools that automate this process here but still it's yet another step in this manual machine learning workflow all right and once all of that is done i can finally deploy my model into your production environment and integrate the api for example into an application to serve predictions all right so this was a very fast walkthrough how a machine learning engineer data scientist would start developing and would perform manual machine learning and all of those steps can take a lot of time right i mean this is not not really done in a single day there's probably a couple of days effort really kind of iterating developing code checking model performance etc all right so let's see how we can optimize this so if we're looking back at our overall workflow the automated machine learning kicks in exactly here so automl helps me to automate all of those steps starting from the data analysis up to the training and tuning at scale and will present you with model candidates which you can then take and deploy into your production environment so in in short automated machine learning aims at automating the process of building a model and in particular it helps you to identify the problem right it it decides whether it's um regression whether it's classification it will pick an algorithm for you it will pre-process the data and it will also help you in finding the right hyper parameters and you might ask yourself well i am an expert machine learning engineer can i still leverage automail or does it make sense for me right so what are the scenarios to use automl and you can really use it whether you're starting in your career and you don't have much ml experience at all this is kind of the the easy um entry point here where you think well i'm just using you know tools that provide with me with automated machine learning and maybe you're coming more from a software developer background or maybe from a business background and then you know maybe your team has the the experts in data science which then focus on the hard problems but even if you're an expert yourself you can use automl to kind of give you a baseline for um you know many of the use cases that are maybe simple to solve but there's just many of them so you can help um get some help with automl and then focus yourself on maybe new problems where you really need kind of the human experience and also just you know help you automate a lot of that work for example the data cleaning feature engineering steps etc and then you as the expert can apply your domain knowledge and maybe do some advanced error analysis all right another important point is the transparency and control so some of the automated machine learning solutions only give you the best model right so you're putting in data and you're getting a model but this is really making it hard to understand what's happening right how to reproduce the environment and this model training step if the only thing that is the the finally um final train model so you really want to check that there's a little bit more transparency and not only the best model return but you also maybe see how the candidates got defined and even maybe get the source code of how the feature engineering was executed and what parameters got set to actually tune the models and then come to that last best model candidate so really helps you to understand how the model was built helps you to explain the model and then you also see how you can tweak it maybe for extra performance all right so what i'm going to show you in the next demo here is how you can leverage automl with a service called amazon sagemaker and here's just a rough overview of all of the aws ml stack and it's really con comprising of three different layers on the top use the ai services which are ready to use so those are really simple api calls into a fully managed service to for example get speech recognition image recognition text translation etc so those you can really embed in your applications without any machine learning experience and on the bottom there is you know the choice of different frameworks infrastructure instance types for the experts that want to customize everything themselves but the sweet spot is really kind of in the middle here with a service called amazon sagemaker which gives you a fully managed platform to build train and deploy your machine learning models and autopilot is one part of the sagemaker family of services and functionality is here so the service gives you really everything from the hosted notebooks to the experiment tracking capabilities data pre-processing training tuning debugging up to the model hosting so we're going to see one specific service here called autopilot which runs automated machine learning and the good news about autopilot is that it covers all of those steps that i mentioned it will identify the problem we're trying to solve it will select the right algorithm it will perform the feature engineering and it also shows you the code that it generates to do that and it runs the hyper parameter tuning it's fully transparent and we're going to see that it's also generating two additional jupiter notebooks for you to use so it's documenting all of the steps that it recognizes all of the findings the insights during this process and generates two jupiter notebooks which you can then grab and use and tweak and develop further if you like and that's of now supporting regression and classification all right i'm probably skipping over this but just want to highlight here's a really really great white paper published by the amazon science team which goes into a lot of details how autopilot works and how the different step actually execute and what's happening there in the background on a higher level what we're going to do is we're going to point autopilot to our data set and it will start looking at the data splitting the data into training validation sets it will generate the code needed to do feature engineering based on an algorithm it picked and then it applies the code it transforms the data and it will also train and tune the model and then present you at the end with a ranking of the best performing model candidates so really really cool all right and those are the steps that i just mentioned so we're going to start with pointing it to a data set which is in a three you're going to configure our automl job with the location of the data and completion criteria we're going to launch the job and then we're going to see the result the candidate definitions and those notebooks and we can deploy the best candidate as an endpoint to serve predictions all right but the better scene in a demo so let me switch back here to my environment and this time i'm jumping into the sagemaker studio ide which is also part of the sagemaker service and here i'm now operating in this nice and i do have here my traditional environments i can see here notebooks which i pull down etc and i also have here on the left an icon for running experiments and this will actually launch my autopilot job so what i'm going to do here is i'm going to click on create experiment and i'm starting by giving it a name so let's call this zero reviews04 and you need to point it to your data so here you can see it needs to be in the three bucket so let's do find a three bucket and i'm gonna take here um one of my buckets i'm in us east and you need to point it to your data set and my data set is somewhere here in the data pocket and you can see here it's the different categories the camera digital software shoes from the amazon.com marketplace so let's use maybe the camera and this one contains a bunch of reviews from the category of cameras on the marketplace all right it's the csv file so it contains um the star rating and the review text and the target is the column we want to predict so in this case i want to predict the star rating and where to start the results so i'm going to pick another bucket here and you can give it for example um auto ml results and this is going to be the bucket where it basically saves the results and you can tell it actually which problem type to solve whether it's binary classification regression multi-class or you can keep it with auto and then autopilot will figure it out itself so let's try this and then you can also run a complete experiment or really just generate insights we want to run the complete thing and you can also specify execution roles with which give permissions obviously you can encrypt the data you can put it in your own network environment and you can also specify the runtime so you can time limit the jobs the important one here is the maximum candidates and this is the maximum list of model candidates to return so let's just say three candidates you can go up to 250 if you like all right and i click on create experiment and this is it so this starts my autopilot job in the background and you can see here it will approximately run for 20 minutes it's starting with a pre-processing of the data it looks at the algorithms and then also um generates candidates and at the end it will tune and then present us with the best performing candidate all right so because we don't have 20 more minutes to wait for the results i've actually run this job just before the session so we can see how the results look so i do have my reviews zero three experiment here so i can describe the job and in this case i actually said give me five model candidates right so i do see here my candidates and i can sort here um by the objective which is the accuracy and i can see here in this just 20 minutes run i already increased on the accuracy from 27 up to 53 and i can further increase by giving a larger data sets adding more data and letting it run for a little longer so i was kind of keeping it short here for the demo but you can see here the different jobs and you can also get in more details about the individual job so you can go in here and you can see some nice dashboards here is how the objective metric the accuracy went up and you can see the metrics that are being calculated and the parameters used right so a lot of insights already but the really cool thing is the additional outputs that autopilot generates and i've downloaded those here so i'm going to open those two notebooks which the job actually generates so the first one is a data exploration notebook and this is a fully functional jupyter notebook right which you can run and execute so in here autopilot describes the data so you can see the input was star rating and the review text and it says two input columns here and it identifies missing values for example it calculates statistics the mean the average etc so it gives you already a good insight in the data and if you have more columns definitely more interesting than um with the two that i have right now and then even better is the second notebook which shows you the complete candidate definitions so this notebook and you can see there's a lot of text so it annotates the notebooks with a lot of normal text here so you can read and understand what's happening and you can see that autopilot identified this problem as a multi-class classification and then also shows the accuracy to maximize as the objective metric and if you scroll down it documents the technical setup the python versions sdk versions it gives you the code to download all of those artifacts the results and the code to set up and reproduce the environment and it also shows you those generated model candidates so you can see here in this case i said give me five options and the dpp 0 is the data pre-processing pipeline this is option one and it shows extremist as an algorithm and then it also gives you strategies to how to do the feature engineering and it does that here with all of the model candidates basically and you can scroll down here the code goes on and on and at the very end it shows you how to do the tuning hyper parameters and also um how to deploy the best model into a production environment and just a quick note here it also gives you all of the the code the feature engineering so i'm just going to open up here this dpp0.pi and this is my python code here that shows you how the feature engineering was applied to my data set in this case it used a tf idf to perform and switch the raw the raw text into word vectors they are really cool and this is how easy you can start an autopilot job with the help of sagemaker autopilot here all right so you can also then choose to deploy the best candidate in production and this is really as easy as just clicking right on the best performing jobs if i choose the job here i can here click deploy model and this will help you do deploy this model into an environment all right so this was a quick um sneak into how to perform auto ml and here are a couple of links for you if you want to get started on the link to our book but also probably more important for you is the github repo where you can find all of the the manual machine learning code which i showed and how to get started building machine learning projects on aws all right and with that thank you very much and i think we do have a couple of more minutes if there's any [Music] questions you