data.bythebay.io: Ruban Phukan, Automating Data Science for the IOT
Recording: data.bythebay.io: Ruban Phukan, Automating Data Science for the IOT
good afternoon everyone uh really very happy to be here uh this afternoon with all of you and to share some of the research that we have done in automating um automating data science primarily for for machine data around the world of uh IND industrial iot primarily and I'll uh walk you through some of those uh findings from our research and some of the approaches that have worked well uh in this uh space and then happy to take any questions uh now let's start with why industrial iot and why is this space exciting uh this I mean you would have already seen some of the numbers being reported by uh different uh different research agencies and most recently the number from Accenture which talks about there is a 14 trillion dollar opportunity trillion dollar economy uh that U industrial iot uh can create right and where is this coming from what are the use cases that we can solve with u data science on Industrial iy the biggest one of course is uh around predictive uh predictive failure predicting failures of machinery and devices why that is important is that it can actually save uh huge amount of dollars for for businesses most of the manufacturers they actually lose a a lot of dollars in uh in being able to predict failures the the second step of being able to predict failures is of course uh being able to perform maintenance more efficiently most maintenance happen currently in a preventive mode which is they follow fixed schedules of of Performing maintenance but essentially being able to predict machine failures you can actually optimize those cycle you can perform maintenance only when uh it is absolutely necessary um the the third one is of course optimizing the inventory of parts that is required to perform maintenance so you don't really buy out all all parts and have everything in your inventory you are predicting what machine parts is going to fail and then you can optimize your inventory as well as the resources the field Engineers that are required to perform uh maintenance um the the the other huge advantage is being able to predict quality issues up front on the assembly line currently most of the quality is identified once a part has been manufactured and it has reached the quality check stage right or it is already out there where you identify that there is a potential issue and you have to recall uh recall the whole item and that caus huge amount of uh huge amount of money uh of course the the the the next step from this is being being able to minimize the warranty claims because warranty is a huge issue if you are able to predict potential issues with the manufacturing upfront you can minimize uh warranty claims and finally prevent uh prevent risk and and security related uh issues which comes with that right um so all of this contributes significantly to the uh to the trillion dollar opportunity uh that exists right but then uh the way that traditional uh data analysis has been done uh by manufacturers uh and and the industrial it space uh doesn't quite work uh primarily there are multiple different sensors across uh different machineries and the devices that are being tracked but they are monitored individually right and then based on the sensor readings um there are uh human generated rules or business rules that are being created saying that okay if the temperature goes up and followed by a pressure drop uh produce an alert right and then you have a team of people monitoring this alerts on a regular basis to figure out uh what's not working so this is something that is this takes extreme uh it's extremely timec consuming it's an expensive process and doesn't really scale right so what is required to make iot work where we are not talking about tens of sensors we are talking of thousands of sensors and if you look at all the connected devices which are out there which is like collectively billions of sensors that cannot be monitored uh in in a in a in in a rule based manner so you need an ability to do it through a machine learning approach where the machines generates what is a potential issue by looking at a combination of All Those sensors and then taking feedback from the predictions which are made out there and uh refining itself or getting better over time uh now let's look at why is data science so hard for industrial iot right the first big issue is is the noisy data right so you have all these sensors which are tracking different aspects uh right from uh different metrics like temperature pressure vibrations water flow you you name it right now all of them are being tracked at different intervals right the failure is not or or whatever is the outcome that you are trying to uh really analyze is not determined by just One of Those sensors it's determined by a combination of those so you actually have to bring all this noisy data together and try to find a meaning of that right uh second huge issue lack of any label training data so you cannot really do machine learning if there is no no labeled data and label data the big challenge is to prongs uh manufacturers are not really collecting this data in the first place they have no means of really providing a clean label training being set for analysis but the other big thing is that you are not always uh I mean the failure is not a binary thing it's not that the machine has failed you need to identify intermediate activities you you don't have you don't want to wait out till a device fail uh every single time you don't want to train a model to only detect failures you want to train a model to detect early signs and that is even even harder uh to do that when you don't have that kind of a training data available um the other big challenge is the models become obsolete really really quickly uh which is where if you are training on samples the number of variations that can happen in production with varant are happening in wide variety of ways it's almost impossible to uh to create one model that will work forever so the models really become obsolet very very quickly even before uh they they reach the production state so all of this makes it extremely difficult to to do this manually it's not a human scale problem uh so one of the use cases that I wanted to quickly touch upon uh before uh before talking about the approach is is a Fortune 50 company that we have worked on this was uh industrial washing machines that we were trying to predict the failures uh of 75 uni sensors each tracking data at different intervals extremely noisy data all the characteristics why data science is hard applies here and and they were able they were trying uh SP they spent almost six plus months to build a model which the moment it was applied on production went from something like 90% accuracy down to 30% right uh and for them to continue to improve was again a continuous manual effort and then very soon they realized that this is this is not something that they can keep doing it on a on a regular basis more importantly it's not replicable they can do it for say industrial washing machines now think of a different device that they want to track they have to keep repeating this process over and over again you cannot hire enough data scientists uh at that rate to do that uh so what's the path of automation how do you really automate that right uh so let's look at each of uh those problems the first is um is the noisy data how do you really solve that uh one big advantage of machine data is that it it has some well- defined structures to it it's not completely random which means that you can do feature Engineering in a more programmatic manner rather than um any other data set which is like human generated where you need to have a lot of domain knowledge when you have to clean the data prepare the data for analysis um second is that how do you deal with the lack of training data is one of the approach that we tried and and uh it seemed to be working uh fairly well is is combining um unsupervised and supervised learning and I'll talk about that a little bit more how do you deal with models getting obsolute really really quickly is to have a continuous learning uh framework a continuous Ensemble of multiple models because you cannot create one model that'll uh work well always and finally how do you really replace a solve the problems of data scientists right is by teaching machines to do uh machine learning automatically that is what uh the field of uh meta learning is and I'll talk about that uh quickly um so let's look at the first problem of uh feature engineering from machine data right uh what are some of the challenges the one big challenge is that you need to get the data into the same scale because every sensor is recording at different intervals it's almost difficult to combine them so you need to Define your time intervals and get all the sensor data normalized or or aligned with the same uh time interval the second thing is that you need to ensure that the data is uh uniform from that perspective where you need to do resampling and impute missing values right so this is I mean uh these are some of the steps which can be automated that can be applied to any machine data problem uh and and and it just works I mean we have uh tried it across multiple different problems um so so the third step is to basically roll up the data for the largest time interval which is there so basically there are sensors which are tracking in milliseconds there are sensors which are tracking in seconds there are sensors which are tracking in uh data in intervals of minutes right so you need to normalize within an interval uh within within one time period which is the larger time period which is probably the minute in this case so that you you have an ability to compare uh the data between multiple sensors the next step is to extract features uh out of the data so this is there are two approaches one is the more standard statistical approach where you uh generate we you roll up the data within within the time interval uh which gives you enough information to capture an event so basically events being uh what could lead to a potential machine failure what could be a potential intrusion what are the series of activities that happens before uh the failure finally happens so so what is the time interval that is required to capture all of those signals is where you rolling the data up and you can take uh standard uh statistical uh approaches of calculating standard deviation calculating Min take the Min and Max values and and do the comparison um finally uh how do you also handle the noisy data problem is by converting into frequency domain from from time series that uh that eliminates a lot of the noise uh so these are various ways where you can generate a fe generate various different features out of the data uh for for analysis uh and and the good thing about this is that you can you can generate any number of features out of it and then perform uh perform something like a principal component analysis or any other feature reduction techniques to really reduce and and do it in an automated uh distributed manner um so once the features have been dynamically identified how do you handle the training data problem so we use a com combination of supervis and a supervis learning approach to do that um all that we get is uh is the feature engineered uh engineered data coming uh coming from all the different sensors uh we apply a clustering uh based approach to identify what are the anomal um anomalous State uh from there and then from those anomalous State not everything is probably an indicator of of failure right and we don't have label data that closes that is there is the prior maintenance records uh which exist uh in in Erp systems across um manufacturing companies so you perform a frequent pattern mining from there and identify what manufacturing activities and which of those anomalous activities are seen more uh commonly in those scenarios and build out a training data set out of that and once you have this training data set uh you can perform uh regression and and get to a model and finally it's not about creating a single model you need to create a Ensemble because it's a continuous learning process you'll never be able to create a perfect model in the first place so essentially you start with the training data train the first model get get the model run it on the real data as it comes in figure out where uh it's failing take those take those fail uh data where the model is not performing and retrain a different model every time and there are of course different approaches that you can take a stack approach or a boosting base approach uh to really uh do this on a continuous basis and every time you are actually adding to the Ensemble so over time you have an ensemble that performs really really well or continues to perform well um finally uh how do you teach machines to do uh machine learning essentially it's about identifying uh the reason it's called meta learning is that you extract meta information out of running various machine learning uh experiments and what kind of meta information that you can learn uh out of the data the first is the characteristics of the data set itself um what what are the different attributes in the data what are the spread of the data what is the distribution and things like that those are meta information from the data set itself second step is identifying for every kind of data what kind of algorithms that they ideally map into what kind of predictive algorithms that you can uh use or you have uh seen work well so uh so it could be that in in some case it's a it's a random Forest classifier that that really works well in other case you it's a gradient booster trees or or any of those or maybe neural Nets that works well so you you extract The Meta information of every run of uh of your your algorithm and then you learn from the base learner itself as to what kind of parameter tuning that worked well for for a particular uh data set right and and uh secondly is what kind of features did did you have to put into the model or which worked well uh for a particular scenario right so all this meta information now is actually gives you a data set on which you can train a model which knows how to apply uh the same approaches when when you have futureed data right so basically that is the way that you can train models uh train machines to uh uh do machine learning uh so the results that happened is that things that took uh almost like 6 months we have been able to reduce to two days of effort where the machine uh increased the prediction accuracy by 300% uh in in like 30 times uh the faster with driving uh cost Savings of almost 30% um that is essentially what we do at data RPM we provide a platform that automates all of this that I have uh talked about um so thank you uh uh and um any questions that you have yeah uh I'm sorry the question is what is the exact kind of problems that we solve I mean the problems is the is the ones that I talked about earlier which is around predicting failures of machines right predicting uh uh predicting quality issues predicting potential intrusion or risk issues on the data so all of this kind of problems is what uh we solve with this approach and this is primarily around machine data yeah yeah um I think we have time for one more yeah quickly touch this at the feature engineering slide but um how so you said that you were resampling to the uniform time series but time serious especially from sensors coming detecting I mean Cal detection you very often get nonuniform very nonuniform corre Ser you have any any recipes or any ideas the command on this highly nonuniform yeah so most of the sensor data is non-uniform which is why you have to go through those steps so we have that recipe built into which which does that so essentially what you do is that you have one sensor which is tracking data at let's say minute level right and that way it's not getting data for every minute maybe there is breaks in between there's another sensor which is like more regular and it's tracking at millisecond level right so one is that even the point at which the data is coming maybe you're getting one data which is coming uh at time T1 right uh and then there is another data point from a different sensor even even they're tracking at different interval is coming at at a different interval right so first step is that you actually identify your own you define your own intervals you say that let me take uh time uh T2 or or maybe an intermediate time period and from there I will divide it into increments of one uh one minute right still resampling this correct so you take all those data and you shift either to one of those you take an average where uh where there's missing values or you take the nearest Value and and apply those so there are different techniques so that is where you apply all those different techniques and and let the machine figure out what worked well by running multiple different uh so which is where it's not just one step it's the combination and the final meta learning step is learning from all these experiments right and then getting smarter and smarter over time correct so it runs multiple experiments right so if given a new data which it hasn't seen before it'll take a longer time to converge but once it identifies what works for what kind of data it'll it'll know what kind of features to extract by default it'll it'll try to extract as many number of features as possible out of the all right thank you very much that really interesting talking if anyone has any other questions hopefully you'll be around sure thank you so much thank [Applause] you