BDSBTB 2015: Simon Chan, DASE – A Design Pattern for Machine Learning with Scala, Spray and Spark
hello everyone welcome to the talk so today I will briefly talk about something extraordinary about machine learning when people talk about machine learning usually you hear about algorithms modeling predictive correlation and stuff like that but this talks a little bit different because we're going to look at machine learning from the software engineering perspective oh wow that's boring but you will see actually we have built an open-source machine learning server to help you build machine learning applications or engines in a very fashionable weight in a way that is scalable in a way that is easy to maintain so first of all very very quick question what is machine learning how is it different from data science so the lot of examples we come of our heads like machine learning is there's something about facial recognition but I'm detection personalization and stuff like that so it's about prediction so most of the time you have some data sets or real time real streaming in data and you need to train a predictive model be it training in real time or training in a batch mode but you need to train something you need to have a model somehow and how do you use a model in your application you need to send a curie either in real time or you pre catch some predicted result and store it in the database and just like they'll talk you just listen to your Spotify they might be doing the personalization every day but you can also do it in real time instantly if you send a curie to the model as well so the concept of having a curie input training model is very important for you to make the decision when you do the software design engine design so what is prediction i/o open source project on github right now essentially we're helping you to build this architecture this architecture is decided mainly for real time production environment real time not meaning real time model updates we see a lot of cases where real time model update is not necessary but sometimes it is but the main thing we see in machine learning is real time curing real time standing in data into the server is very important so on the right side is your applications and nowadays most business you have more than one applications it can be mobile applications can be video it can be email data it can be wherever they're basically you have multiple data sources right and what is friction is on the left hand side it's a stack of machine learning processing stack basically it comes with an event server underneath the storage can be HBase can be my Seco can be post quest can be West shift so anything but on top of that we built a very thin layer rest layer for you to help to help you listen to real-time events and that layer is built on the spray framework so prediction out the whole project is built on scholar by the way of course that's why I'm here so after you the gate data is streamed into the event server on the on the on the stack you can build one or more engines so will the main focus of the talk is about the engine so how do you build reusable easy to maintain engines but what is engines each engine represent one prediction problem so in the same stack in one set of data you can have for example personalization engines you can have current prediction engine you can have fraud detection engines so because you have one set of data it means that a lot of components should be reused that you should not be reading the same user behavior you should not be writing the same code to read the user behavior and and copy it everywhere in different engines so reusability is very important in real life production development so if we look at engines level what prediction out gives you is respiratory of open source template some of them developed by the core team some of them were contributed by the community so there's a template gallery where you can go and see the source code of different types of application engines like fraud detection Sharon detection recommendation this one the power of engine is that engine template is that their template so it means it's customizable supposed to be customizable so if your scholar developer you can go and take a look at this code it they're all built under the same design pattern the dasd design pattern and so essentially you can modify the algorithm you can modify the business logics by yourself so one recommendation engine template you can actually use it in different scenario some people will use it on the front page of the web some people will use it for the mobile push notification and even though it's the same for your same business but you can see it the algorithms or the goal you want to optimize this very different right like on your front page let's say like a music service front page the goal may be about discovery may be about finding new content but if you're doing push notification the same service the go is about getting the user come back to the surface so it's so they're two goals you want to optimize but in a coding level a lot of things you can reuse really some same to text classification you can use it for spam detection you can use it for sentiment analysis so going down into the code level quickly each engine contains four components I'm cheating actually to give it a nicer name dasd but actually d4 present two components data source and data perforator well you sent you that dealing with data remember you stream in your data into the event server you can in each engine you can modify the data source component to pick the data you like from the events store and you can have the day separator to do the feature extraction freeze your transformation to prepare the data so that they can be there ready for the next level algorithms or algorithm or algorithms so the beauty of this architecture is you can start with the most simple algorithm in the world let's say random or like random result or show just the latest result you can have the whole pipeline ready so there's kind of like a dummy engines the prediction is rare dummy but you can have the whole pipeline ready first and you confirm it with your product development team with the business team before you put in more complicated algorithms and you can benchmark different algorithms to do evaluation so one engine you can deploy multiple engine instances so one may be using random forest the other one may be using random 4s plus for example collaborative filtering combine them together to get results so you can do AP testing easily without rewriting most of the code you just plug in a new algorithm into it the front layer is serving so we remember we want to care about the real time kure right like so music surface so i'm in oakland i send you a curie now in the afternoon so you might want to send me some really upbeat music whereas if it's at night somewhere else you might want to send me some really quiet music so that that is something not handled by algorithm or the model itself but more on the real time real time logics real-time layer so serving is where you put the real time business logics and evaluator not going to cover too much on you evaluated today but you can't do it so the two things i want to show you how you can visually to show you not showing you how to build a code within like 10 minutes but i'm showing you what you can do where the engine is two things one is train model the other one is respond to dynamic curie so training a model that's the main go when you're doing machine learning right so in one engines if you go download the engine temp light source code you will see this classes a data source class a data pepper ater and algorithm so remember data source data represent for d & age 94 algorithm stand with a is 94 the algorithm so how it works is you run Pio train essentially you training the engines with the data in your event server perio train will trigger the retraining function in in a data source class and whatever you like to do as long as you output the training data class objects sorry that that's fine so training data will be passed by the framework automatically to PAP rated class and you can do whatever a feature transformation there as long as you return back prepare data so prepare the framework will help you pass the prepared data to one algorithm or multiple algorithm classes depends if you have associate one algorithm or multiple algorithms in this engine so let's say you only have just one algorithm 1p per data will be passed to algorithm one and now you can have your algorithm logics how algorithm code in it and create a model and it can be your DD format it can be data frame format where whatever it is so um frictional architectures assuming you're using spark you're using scholar so you can you can return the model the model will be will make persistent automatically by the framework that's all you need to do for training part you don't need to worry about scalability you don't need to worry about when you want when do I train it how should I train it because this is something configurable on the server level like as a developer you don't need to worry about all these logics in your code so a visual way to see it because I really want to show a demo but i think it's tough to show a damn awareness a software development platform so let me try to show it visually again we play the scenario basically free laughs over stuff datasource passed the training data to data preparador data preparador pass to prepare data to algorithms algorithm produce the models system help you make them persistence simple and easy okay now you have models right and you can define the times you want to retrain these models the next thing is you can deploy this engines on production as a web service with just one command line so as they now your engine is running as a web service what's next you can then send rest curie in real time to this engine web service what it does is it will trigger the algorithm class this time is to predict function instead of the training function the Curie will be passed to it and you see that you have access to the model that you have trained before so now you can do something with the persistent model with the curial of course like this to make a prediction and you return the prediction predicted result will be passed to the serving class to serve automatically and this is where you do the real-time business logics remember so if you have multiple algorithms the serving class besides doing the business logics it can also do the assembling of the algorithm results like it can do a voting it can bore do it like voting by majority you can do combination is going to do a great angle in there you can even do a linear regression in real time here as well so visually looking at how it works you have algorithm algorithm one you have serving components your mobile application send a real-time kiri to the engine surface which trigger algorithm one which contain the model and it produced a predicted result which will be passed to the serving and surfing we return the result to the mobile application as a as a JSON as as the through the rest call so that that's how it works that's how you can easily create a machine learning surface with reusable components which support multiple algorithms which is for plug and play however from space and components as well easily using prediction i/o framework and of course like that you can you can have more than one algorithm here so the code to put all these components together is really just like one factory object you point to one data source do plan to one pepper ater you yeah we you can define a map of algorithms and also you point your class serving class so running it on production is is easy so install friction io the whole stack that i mentioned we're sparked with buildin's build on scholar with spark with HBase with the rest built on spray just one command line everything will be ready and of course it's supporter Custer setting as well so it's listed on amazon marketplace things like that so starting the event server run one command line you can download the engine template from the template gallery and running it is just another is cheating it's actually freaking my life but I put it like one come on but it's 11 line of commands and you can update a model periodically or whatever you like so the server also provide other features which I'm not going to cover today but versioning rolling back the algorithms model like if you're training the engines every day and suddenly something's goes wrong you can roll back to the previous version of previous days offline on that evaluation is something really fun but more difficult to demo here getting multiple engine variants and Kure and production tracking this all like comes with the server and it's all open source hundred percent open source under apache license and we have been working hard on a new recommended template we call it universal recommender not really good at naming but as the name suggests we try to have a template that can take in multiple signals it can be user event signals for collaborative filtering it can be trending signals for something more real time so as a one engine template with a lot of things combined in it for many use cases so it is actually a fun to look at it so this is our website and if you have any questions let me know I do have some stickers here so if your fancy with a sticker also come by thank you any questions yeah I yes please yes sure sure we love to actually the project start of two started two years ago at that time spot was really early and we support hundred mapreduce basically relying on the mahouts library so we're not like strictly only supports park but we would love to support other other stuff as well and we have some contributors who have created engines running on for example h2o algorithms we don't see any case on Frank machine learning library yet but it's actually very self is actually something very exciting Thanks any other questions oops so so the question is about the speed of respond or the latency when a mobile application sent a request to engine web service and we need to trigger multiple algorithms multiple model and combine the result and return it so the latency can be very long yes so the burden of latency is actually on the software engineering side but not so much on the frameworks I what the framework does is first of all is distributed so it means that the algorithm the model itself is loaded in memory in multiple knocked in the server so loading them you're not doing real-time model training is essentially just like trying to make a prediction and even though you have multiple algorithms model models you're sending the request to all of them at the same time so the time no matter you calling one algorithm or multiple algorithms the time to get the result is as long as the slowest model to make prediction basically amongst all your high waves and combining the results on the serving layer then it really depends on how you design your serving layer if you're just simply F reaching the score or combining the score then it can be fast but if you're serving layer contains some logics to look back to your database finding some unseen products itself for example that it would depends on your the call to the database basically that part of the speed thank you alright thank you