Simon Chan: Q&A with Alexy Khrabrov of SF Scala @StumbleUpon 20150122
Recording: Simon Chan: Q&A with Alexy Khrabrov of SF Scala @StumbleUpon 20150122
hello everybody uh i'm alexi krabro the organizer of sf scala and today we're on location at stumble upon where we have a meet up with two talks uh prediction ion stumble upon itself and we have here simon chen uh the ceo of the company and he's going to be doing the talk so i'm going to ask you first of all what is prediction prediction io is an open source machine learning server it's built for developers and data scientists so the goal is to help them build machine learning applications much faster than before probably in a fraction of the time and deployed it on production so that's the goal of prediction io and it's written in scholar and it's open source so is it is it open source package which you deploy on premises you can integrate with your application sure yeah so everything is on github so you can download it it's built on top of apache spark ml machine learning libraries is integrated in it and it's also covered with xbase so you you can download the whole package install it on premises but we also support um private cloud installation so if you're fans of like aws or xbase you can also deploy that on those private cloud services okay so what is the business model behind it um yeah many people ask this question it's like are you guys a charity giving out the source code on one end we really like the product to be used by a lot of developers so we make it freely available so developers like you and i and everyone else can use it try it for free at the same time actually open source business model open source software model has been around for many years if you look at red hat my sequel mongodb um or even like hydro clouderas companies like that so um we're pretty much following that direction if you find the software useful and if you want to like use it on production then we're very happy to find professional support and services so it will be basically so the all the functionality is there from the day one but if you run into problems if you need help with specifics of the product in some extensions you need sla and support you will basically engage you definitely and machine learning is still in this early stage unlike the host like stumble upon maybe they have been using machine learning personalization technology for many years but still a lot of companies they're just at the beginning so um i think it's very important at this stage to make the product useful widely accessible so people can use it and we have a quite active and healthy contributor community so hopefully we can gather the forest and build a great product together okay interesting so you know spark is wildly popular it's like it's one there's a phenomenon and you know one of the great tools it's a great rule right it's one of the biggest uh communities on github one of the biggest meetups in the world uh and actually this you know with spawn of spark made up of scala and it's almost you know one and a half size of it two years later uh and uh i wonder so if you integrate the spark right this kind of building applications on facebook in some way right so basically how do you kind of think about the dependency on spark you know spark is rapidly evolving itself uh so what is your thinking about building products which integrates with spark sure um this is a really great question we hope to tightly integrate with spark so in fact prediction io is one of the few products that is certified on spark if you're running spark on production and if you want something a software platform that is certified by professional people of spark prediction is one of your options spark is evolving but our engineering team is also working with their engineers closely to make sure that we're compatible with the new specification and one of our missions is actually to take care of the machine learning side of spark especially on the production deployment side so developers data scientists who install prediction io will automatically get the benefits of apache spark without even learning about it or knowing at existence okay so this is basically extra service which will simplify the usage of spark for for designers you can think about the relationship like um might not be the best analogy but kind of like wordpress and mysql you use wordpress and that's my sequel behind it and power a lot of stuff in it so we really love the apache spa community and hopefully we can be one of the contributors in the journey that's great so uh tell me a little bit more about your own motivation so you know what brought you to machine learning you know what uh like what do you know what brought you to to make prediction io and you know why did you choose scala and spark to do this yeah i built three startups products in the past many consumer facings products so even though they are not machine learning or artificial intelligence products they touch a point personalization recommendation predicting user behaviors this kind of stuff and i felt the pain that every time we need to do something smarter it takes our engineering team a few months of time to do it um it is like oh if we need data storage i need to build my own machine learning server in-house which doesn't make sense right but in the past like 10 years there's no solution like that exists we have a lot of great tools spark hadoop but still there's no nothing like database server for the machine learning community and so um i went for research study in machine learning for a few years um didn't know what i would expect at the beginning but the more i look into it i realize that there's a big gap between software development of machine learning stuff and research stuff there's a lot of great things going on in the development side and also the research side but prediction now is trying to fill the gap so that developers data engineers can make use of the software platform build machine learning stuff much faster it's actually start off the idea as trying to solve my own pain and also my teams co-founded their own pain they're trying to build something like a restaurant recommendation app and i was lucky enough to convince them like hey why are you doing that why don't you build something for people to build the app faster so that's where we started this is great so now i think i better understand this so this is really interesting especially commoditizing machine learning right so you should have the same ease of use for machine learning as you expect from a database right because you can switch from sequel to pause grass without too much thought or oracle right like if some kinks will be different but it's not like a radical difference up you don't know where is what right so in the same sense uh at some point machine learning should become as kind of understandable the data pipelines like what ml lib i think uh a roadmap for this year is create these data pipelines where you know tokenization and and model right and prediction come become kind of understood steps right so so so this uh service hopefully will enable people to kind of take an expected stack build their machine learning quickly and focus on the actual problem that they wanted to solve so it's like in the web development community right um developers the developers in the company should focus on the unique business logics not the web framework at the back so we're kind of like working together with the open source community spark mle trying to be the backend for developers excellent so maybe we'll end up with kind of a prediction for prediction i also where do you want to be in the next few years what do you envision what do you want to take kind of most help of the community where do you want to make most of the contributions can you outline the vision sure machine learning is still very fast growing evolving um community as you have mentioned so um there are a lot of issues we still haven't solved one of them would be how do you find the best model for your application for your unique data um what's the best way to serve developers what do they really really need when they try to deploy machine learning applications on production so we need a lot of feedback from the community and early users to tell us hey it doesn't work it works and in fact we're getting more and more github pull requests to the product to so we're improving the product step by step and day by day so we started off the product from the scholar so it's great to be in the sf scholar community sharing the product and hopefully scholar developers especially um would take a look at the product and see how they can use it play with it and give us feedback because scholar is a beautiful programming language but at the same time it's not the majority of developers knowing how to manage scholar at this moment but this group of developers developers who are into scholar right now they are the pioneer they can push it forward and we need contributors like that sounds great and yeah i said this last question but you you know you kind of made a point which begs for kind of one more follow-up question uh what do you think of the potential of data science coming to scala platform because currently a lot of data scientists work in python or are and as a community we know we have the power of distributed systems we have functional programming we have the performance what do you think are the good tools and ways to kind of inspire uh data scientists to take a look at jvm and scala and prediction io it's very interesting questions we have seen a lot of debates in in the past years ago hey python ruby um php java and i think that the choice of programming language really depends on your preference we totally see the value of jvm especially on the production side especially on the type safe side so definitely if companies are building mission-critical applications they want maintainability or having a large team working together on a production mission critical applications jvm language would be a good choice and scholar is like way cleaner than what other jvm languages would have offered in the past and if you look at the syntax it's actually like the native language for data scientists exactly like the functional programming so you can write distributed stuff yeah it's like native so why not start with scholar it's great and you can use all the job existing java resources right you can incorporate apache major libraries into scholar program like easily like natively so i do see a great future for a scholar um and i think this python and scholar community will grow in parallel i believe sounds great well we'll we'll see how it pans out and thank you very much we're looking forward to your talk thank you alex thanks you