scale.bythebay.io: Yuhao Yang, Bringing Deep Learning into the Big Data Analytics
Recording: scale.bythebay.io: Yuhao Yang, Bringing Deep Learning into the Big Data Analytics
you yeah my name is Joo ho-young and I'm from in hell baited had homology Department actually well now I'm dual parading the big deal team and before that I was actually focus on contributing to a party spark and I contributed like linear ICC and RTA and I figure of sir algorithms in spark so now we are focusing on big DL which is a deep learning library on spark ok so race big DL so basically a PTO is a deep learning it's actually a nerd a deep learning library on spark we built the library in her in her Liam Scala and we provide Python interface so that's where big DL we locate in the software stack we are basically in parallel with sparking our life and we have provide provided to interface Faust but I'm not apply so you can integrate it with you so you can integrate speak ETL functions and say cue and streaming and I'm alive so about the basic components so for building a deep learning librarian spark we have to provide a few elementary elements for any deep learning applications so start we call tensor which is basically a multi dimension matrix a restructure kind of like the end array in Python and this generic data type currently with supposed float and double so the key point about the tensor is that all the P Ras operation we handed a back to Peyser Intel MKL library I'm Cal stands for math kernal library and with the help of mkl we have comparable performance we GPU on certain actually many different applications and actually if you help you die native renewing applications you know no matter is like an action our convolution or and at the career part of all the different networks that cooperation is ecology and ma'am which means multiplying two matrix and by leveraging I'm K out multiplying of matrix can be accelerated by orders of magnitude magnitude so after that we provide layers here we Imperial currently we provide over 200 different layers including the common collusion arlynn and freely connection and all kinds of criterion so for the criterion based ladies for different types of deepening tasks for regression or classification and for optimization we provide stdio PFDs atom at the booster and basically a recent popular in the model two we help you well here's the list of the models that we have implemented on PDL so Alex tonight inception twice night we DG and mobile night for image classification SSD faster RC n for objective - and wide and deep learning new york recommender tip a time is for recommendation and we have provided some industry applications also in the models who awoke you more details later okay so what seem big dl2 summary up we have Scala and Python interface and we have build a lot of applications which can integrate big DL and spark ml as back streaming and the interface that the programming interface is actually quite similar to spark so a nice part developers should not should fund in T we should find the API pretty intuitive and we also provides a part of our Jupiter notebook and visualization through tensor board you users can also load cafe total tensor flow models into PDL and to find union after that and another great thing about PBL is that we well I talked earlier about mkl and it's actually a library written in C so we have wiped the C files and make it an independent jar file so when deploying PBL you don't you just need to copy the jar file for ppl along with your application code to the cluster so there's no operation actually there's no any operation required for deploying big the applications just like just like super meeting in normal spark job okay so we help while catch the spot from all the cloud out the primary cloud providers including either AWS and Ali and cloud era rotate a breeze so they have all published blogs about how to use PBL on their platform and PBL was open sauce on December actually December last year so current lays crackling over 2900 stars and we got a lot of contribution from the community right now intel has a developer team about 30 developers focusing on the big deal development so we are pretty committed to making these projects as successful so who is using big deal and we are actually cooperating with a lot of our partners in different Harriers for consumer sites we got a space and MLS listing and they are basically well I will talk about this I to use cases later actually chaos space use is using PDL for speech recognition I'm a mouse to these scenes using it for funding similar homes for the users and UCSF is using it to precise it 3d images from MSR and from finance we deployed big deal in Union pay for its fraud detection system and for China life and master we build recommendation system for it and JD dot-com is a highway user of PTL actually they cut a cluster a spark cluster of thousands of nodes and they found a big actually they replace that their deep learning they replace their GPU based at deep learning classroom with big deal on CPU because they found well it's just a more natural a to derive with deep learning applications on the data that's already in the big data on their hadoop clusters so using PBL has several advantage first you get yourself well most of your beauty has often engineers are pretty familiar with Hadoop and spark and now it's api's so you don't need to to get here developers to learn is he knew for running deep learning applications the second thing is most times your data is already in the big in the big in your cluster like Hadoop Hall HDFS or which base alka Center so by using spark and big do you can just access your data in our table a and the third is actually for big DL well it's much much faster with other a compared with other deep learning library because we have i'm carol acceleration and actually it's the target of pikiao is to get comparable performance with GPU cluster okay so next is pasta and he's from steel manufacturer and he is using big deal to detect his product defects and creates a another platform provider for science computing okay so next i will go into more details for all the four well each of the industry application as an example and i will try to tell more details about each of the application which kind of algorithms we use there and what kind of results we catch so first day is from Teddy calm to user users mirror is pretty straightforward it's just to find similar products based on images so to do that the first type is to detect object from an image like initially we need to precise housing images and then we need to send it through the ICSD model so we can detect certain image height from the image so we can filter out all the background noise and after we got that part actually it's a crop from the image we can use that to catch it through the deep bit model which will calculate the binary a binary representation for the image the panel representation can be used for image similarity search or just the image quarry and result is caching HDFS system the next case is a Simo about a pre-processing part actually big deal focusing on providing end-to-end solution for the users so we have provided all the necessary types for the image argumentation and to stop defending here like the image transformer is actually extends from spark transformer so you can really fit it into any spark I multiplied already have and we provide all the types whether you can just define the parameters so for next example is from real estate basically if you have searched the house in the Bay Area the agents will upload out metadata of a house like the attributes like sites the price range and how many page rooms and other attributes and they will also upload a bunch of images including the interior and is the exterior of the houses and your lease nothing here you see the image of a house you can decide like the house on out so MLS listing actually is developing an application to fund similar houses based on images with PDL for this project is actually a kind of hybrid system where we build three image classification tasks first well we try to well classify each of the image into two-tailed to add the text to each of the house like whether it's a ranch-style house and oh it's a contemporary or it's a traditional style house so there's a style tag another Tyga variety is the story tag how many layer how many stories are there for a house is a single story to story Austrian plus and with those types so we can get a basic classification for the house images and for the similarity part we actually use image embedding from the vgg Network we use the vgg 16 here and we use the first linear layer that's immediately after putting five layers of the VTT network is a 12 is about 12 or 25 thousands of flows for if you generate about 25,000 whole flows for each of the image and we use that as an inviting folder for each of the image and compute cosine similarity to calculate a similarity of each of the images so yeah that's a panel is also in the final result we combine the tags and the similarity image embedding to calculate the final similarity okay nice example is from Giga space well also example is basically just for its full customer service support and basically customer cost through the phone he can tell you about briefly discuss describe the problem he has and then we will use speech recognition to convert those speech into a text and we will use text classification to classify it to actually rotate to a special Hillary College well a specialist in a specific area like a user if he has some problems with him with his mic then we will just transfer it to an i poll specialist so that's the basic idea in this application we use the post speech recognition and taxol master technician and the link the link below is a bigger space blog about how to build it here is speed here so part of fraud detection use case Sheen UnionPay well this was a long project and it was actually divided with over three menses and the first types are well collecting data and to some mini show analytics so we can generate some extra features like how many how much money has a user spending in the last three menses and was the last location of his transaction and after that we feed all the data into a feature engineering pipeline where we do some resembling and imputation and indexing the resembling ET is specially important for the fraud detection use cases because in any fraud detection use cases the Trinity will be highly imbalanced your letters are only two for all transactions in every 10,000 normal transactions so you have to here we use the resampling to resolve the well implant dating violence and for the model part we actually used in sampling of neural networks earlier we tried with tree based models like random forests lgbtt but turns out well in assembling of neural networks get a better accuracy for this one and for each of the model at a small neural network in the in sampling we actually receive both the data from the original data resembled a part of the data from the original data so actually each model will be feeding different training data and either for the protection of the neural networks will come together and vote and to see if a transaction is a fraud or not and here's you can try with different voting strategy to see which gives the best result we use the grid search actually an improved grid search from spark in our life to do the parameter tuning okay so next and is for China life insurance is in your recommender example well the network is a pretty straight forward actually for each of the user an item pair in the well training data we will try to extract embedding for each user and each item so quite similar to other relative learning patient recommendation energy and yeah with the the lookup table you can efficiently catched inviting for all the users and it can be used in multiple scene errors besides you know in recommendation you can find the similar products or find similar users and the model party is quite straightforward yeah we got deep learning and depart and GF part also well I didn't include it in the slides but we also support wide and deep model in PDL and if you try Katie you can check it in the analytics to of big deal okay so here you are welcome to partner with us actually right now we are still promoting big deal to the industry so we can provide actually Intel that not provide commercial solution like cloud era but we can provide support if you want to use big deal in your application so we can help you develop it and provide some consultation so welcome to partner with us and the following links are the links as the bottoms are the github repository for PDL and some use cases from inter-site so thank you everyone that's [Applause] it's a mandatory question of how to compare 2004 okay well in terms of function parity we pretty much reached the same level basically I read every model that tensorflow can support we want to support it in big deal and we are also actually PTR just release is 0.3 version and you can just load all that has a flow model you want not only the tree in the model you can actually load the model the definition file from tensorflow and load it into big deal to continue training so we support how we try our pasture to support how all the tender flow users and we actually want to help you to migrate to pl only our spark cluster and in terms of performance actually we tried several kinds of deep learning applications and the target here is really to compare big DL on CPU cluster which tensorflow on GPU clusters so we can really be compatible in the market and from my personal experience we most times we can get comparable performance sometimes we can get like 80% of tensorflow on GPU cluster sometimes we get a little better than them actually JD delcom tried it and it funds that baby elk actually runs three times faster and has a flow under GPU cluster so they switch to a big deal and so okay for most of the production development I see a lot of usage for Scala interface well most times because while Scala is more type safe and easy to test and thought their production environment your lay they do not allow any change - there's clustering environment so they would just deploy the scholar base to speak the applications because it's just as you know deploy any deploying any standard smart programming it's just as just as a jar file and you don't need you don't need to change anything on your cluster I were to run Python on your class or sometimes you need to install Python one each of the work notes sometimes it can be a blocking issue and but for you know prototype development and the data scientists that we do see a lot of people using took their notebook so we are supporting that [Applause]