SBTB 2015, SF Scala @Nitro: Marek Kolodziej, Scala, FP and Spark - the Perfect Combo for ML
this is not particularly sturdy alright hi everybody so this talk is more about just scholars and ecosystem crap okay this this is more of a talk about scholars and ego system for machine learning in general is sort of a survey and overview not so much the details of any particular algorithm like Lda but I guess I'm just the main point that I'm trying to make is that even though Scala may not quite have the same ecosystem that Python are are have it's definitely good enough start and it's definitely much more production ready and especially when executing at scale so first I'm gonna overview what we might or might not be missing from our in Python what the JVM alternatives are and hopefully scout alternatives so as not to have to write boilerplate Java code and and actually I'm going to show how to develop a really simple sort of toy example algorithm from scratch in just a few minutes so first of all you know obviously machine learning is ubiquitous it's everywhere from fraud detection and spam filtering to speech recognition OCR and even self-driving cars but in order to support so many algorithms you actually have to have tons of models and these are just a few regression models they're actually probably you know two at least an order of magnitude more algorithms for regression itself a budget classification algorithms again the list is way way longer than this sing with clustering you know it often times you might you might need something less vanilla than say k-means for example you could do clustering using self-organizing maps which is a neural network algorithm essentially you need a variety of algorithms to solve a variety of problems and in that case you would rather have a bunch of libraries so that you don't have to write all of them from scratch because some of these algorithms are quite complex and quite tedious to implement them and especially to implement efficiently and implement officially its scale so in the case of the our ecosystem you've had this exponential explosion of the number of libraries there over 5,000 libraries and are right now which cover everything from the design of experiments and traditional statistics to machine learning you know and I'll be obviously plotting and a lot of other areas Python obviously is quite rich in that respect as well from from linear algebra numpy to plotting in my fluid machine learning and so I could learn even symbolic computing and sin Phi which is something that only octave and matlab offer other than other than Python but you also have some really advanced solutions that are unmatched in other languages such as Theano and pi brain for for deep learning and and and so this ecosystem is quite rich as well I mean the testament to this is the fact that you can write a digital recognition algorithm in Python in 30 lines of code and in fact out of that we only have one line to train the support vector machine classifier and one to predict them the rest is just visualizing the output on the console and and that just shows you how powerful these things are so essentially this is the kind of level of support that we're aiming for in Scala which and it's quite not quite there but at the same time you know as good as Python and R are in terms of the number of libraries obviously they have a lot of problems and my least favorite issue about these languages is that are strongly typed so I can just have a typo in the code let alone a type error and and have it fail at runtime after a few hours simulation because that particular path has no they've been executed before and that is really painful and the runtimes themselves are quite terrible as well like everything is by copying R so you waste a lot of memory the garbage collector is really slow Python has the global interpreter lock so you program as if you essentially had to to scale through to multi processing on it multi-core machine like it's I don't know CGI web programming in 1995 sorry did you have did you have a question oh I think I mentioned did I not mention that uh so I did mention pandas yeah that data frames for Python yet right that is a good point that is a good point our has some workarounds are actually has some really clever tricks to do a lot of really weird things like for example you can inspect the previous stack frame and figure out what the variable names word that were passed to this to this particular stack frame and stuff like that so they do have a lot of crazy things that you can do you have actually a full control over the interpreter right right right so so it is it is nasty and you know and the way obviously people scale are and and and Python is that they write everything in C++ or Fortran that have a wrapper around it right in order to scale but that's not quite nice you have to stew languages and it's a pain but you know the other thing is of course that there these languages encourage imperative programming and math is declarative you know and I'm not just talking about piano numbers but I'm talking about the fact that for example the output of the last layer in a neural network is dependent on its input and the input is dependent on the output of the previous layer so you can essentially write everything as one big recursion so you know it's more than just like stylized examples like Fibonacci numbers everything could be potentially recursive quite elegantly and you know no side-effects in math obviously you know people say that R is actually a pretty pretty functional language because you can have a function declared within a function and you can pass along doesn't stuff like that but you know if you don't have tail recursion and obviously don't have you cannot write much using using the function functional paradigm in R and Scala you obviously can so you know how does the ecosystem compare because sure we're purists here or semi purists at least than we believe in FB and we believe in static typing and so on and the JVM is probably a better runtime but you know if you don't have any ecosystem then you know people are not gonna give you all the time in the world to write the libraries because you have to deliver results right it doesn't matter what the run time is at that point so I just want to make the point that the Skyway ecosystem is actually not not that small so for example for numeric abstractions you have axial and spire spires really phenomenal these guys have come up with really crazy things such as a floating point filter to have the accuracy of essentially a big decimal and and while while keeping the performance of a double yeah I mean I don't even know how they come up with these things but it's really cool a lot of nice category theory and stuff like that as well if anybody knows the difference between a rig and an RNG and their ring then bonus points for them okay but from a more practical side here sort of a higher level of abstraction linear algebra is the bread and butter of machine learning and it would be a shame if we didn't have good libraries so the three libraries at the bottom are actually Java based but they're api's are not that bad and e4j has nice backends it runs you know on CUDA and and blahs and all kinds of other backends but unfortunately still has a mostly Java API there's there's MD for s now for Scala which which is a major improvement breeze which is the first logo by David Hall this is part of Scala and LB and and it's a really phenomenal linear algebra library that I use all the time so there are lots of options and obviously the notebooks and MD is here so this is my favorite of the notebooks but I listed all four and there they're plotting solutions the notebooks obviously have plotting solutions on their own but there is also wisp for web-based plotting there's a lightning visualization server which is language-independent it has a REST API you can use it with anything but even breeze has visualizations although those are based on J free charts so not as nice but breezes focuses on linear algebra not so much and zatia and with mushiya with respect to machine learning you know you have a lot of machine learning and breeze there is also stuff in Factory there's there's Anna live of course and and h2o Fork of spark as well and deep learning for J for deep learning you know these these are somewhat deficient the other solutions you know that I just mentioned for for basic sort of vanilla machine learning but where the JVM really shines is an LP and at the bottom we have the three Java based libraries including Stanford Cornell P but that the one the three ones at the top are all purely Scala base so we have factory Park and epic and park actually does parsing on the GPU so it's really fast so you can have performance and elegance at the same time but what if there is no library for something right I would like to make the claim that it's really easy to build it in a scow just because it's a good language so you know obviously we have functions as first-class citizens and functors monads and you know the whole category Theory shebang and you know not going deeply into category theory I would just make a Bateman a more basic point such as you know referential transparency matters you know and and immutability matters as well and and just the fact that I can do declarative programming and not get a stack overflow but also Scala has implicit and this was the point that Malcolm was making about type classes it's really nice to be able to write one algorithm that will run on flank Scala collections are a spark rdd's etc etc and of course you can write nice dear cells because the language is very expressive and you know just the fact that you can have you have type inference to don't have to write a lot of boilerplate for the regard to the types and you don't have to write the Builder pattern because you have named arguments to tier two your constructor for example that's pretty nice so just the language is really neat so just in a few minutes actually less than 15 minutes I'm gonna show you how you can create two optimizers stochastic gradient descent an autograph using breezes linear algebra developed linear regression from scratch by implementing the cost and gradient functions have the implicit conversions in place so that you can do the computations over scour collections and spark rdd's what the results and run against the synthetic data set and actually I wanted to make a special point about this abstraction over scale collection as a spark rdd's is that the point here is that sure you can make a conversion to an RDD by paralyzing the scale collection using the spark context but the problem is that if you're running in a single machine and you don't have much data spark or actually slowed you down because you know it's doing scheduling and stuff like that and if your data is small you might as well run faster using scale collections and this is the point that data bricks was making that project paxton isn't just about cogent and like native memory use and stuff like that but it's also there there was a plan and the next couple of releases to basically have local versions of the libraries that would that would run without the overhead of spark itself but it's really not that hard if you have the kind of type class that Malcolm was talking about so linear regression you know before we go into the code you know you have you have a feature on the x-axis and you have a target and on the y-axis and you're trying to predict Y as a function of X obviously this relationship is not perfect therefore we're trying to minimize this sum of squared residuals and we could have many more features than just one but in that case we wouldn't be able to visualize this so this is a really simple model and and the linear algebra is really straightforward you just have your beta matrix beta beta vector which which has the coefficients for the features and your X matrix where each row is an example and and each column is a feature for that particular example so that should be pretty straightforward to implement now with the regard to optimizers there's the gas degrade in the center out of grub here normally stochastic gradient descent is just represented as performing the computation on a mini batch or even on a single example in case of online learning and in that case you calculate the gradient for that particular example or the mini batch and subtract the learning rate times the gradient however stochastic gradient descent converges poorly unless we were actually doing learning rate and kneeling which is what I'm mentioning here in the case of autograph instead of annealing by the number of iterations and the number of examples per mini batch where we're doing that based on the the norm of the gradient history for each dimension so the bigger the gradients in a particular dimension the less you learn in the future so it's kind of like the authors of autograph essentially summarized that as tf-idf but for optimization we have the cost function which is really simple sum of squared residuals and the gradient is really straight for it as well if we just think they're a derivative so to actually implement this really quickly I'm just gonna skip this step with type classes because Malcomb already covered that but I'm using the same thing because we're doing this as part of the book that we're writing together about machine learning for spark so he already did a very good demonstration so I'm gonna skip that but let's just look at a couple of other things first of all the optimizer is going to simply take some some hyper parameters here such as the number of iterations and the initial learning rate and the momentum momentum is a way to preserve preserve the the direction in which you moved in the previous iteration that reduces the variation in the direction in which the the movement is unstable and boosted in the direction in which the gradient is small but the direction is stable over the last few iterations and for the update function this is actually where we provide the weight update so since since since Scala's functional when we can and functions are first class citizens we can basically provide provide a function object here which defines our particular update rule which is different between a DeGraw data Delta as GD v SG D and all of the other optimization algorithms and and all we do here essentially is is executing the cost function which we already defined for linear regression at least mathematically and then we get the we get the optimization history value and then we call the the update function a bunch of times and we don't even have to write a for loop because we could just default so since it's Scala and now we actually need the optimization algorithms because this is actually the key issue right we need to we need to know how to update the weights so this is all of stochastic gradient descent which will actually run on on both SPARC and Scala collections at I grad is a little bit longer because it has this step where you scale by the diagonal of the Hessian which is basically the the the accumulated gradients on on the diagonal and and and and so it's a little bit longer but again very very little code right you need to do some sampling and this is important because we're doing many batches and and if we were to do full batch gradient descent it would be extremely slow with many batches it's much faster however it's still faster to do many batches than to do pure online learning because we're using vectorization and actually this is an important point right the JVM is really horrible at optimizing numerix because first of all for a lot of data types other than arrays of primitives you know we have boxing and then and then you then you lose a lot on object construction and destruction and and on basically dereferencing these these these references instead of having cache locality and so on and so forth but also the JVM is really bad at a vectorization vectorization was introduced in Java 7 update 40 or something like that so fairly recently and it only works on basic Java foreign while loops but if you have anything more complex and I'm going to work and it doesn't even support recent vectorization sim D features of CPUs such as you know AVX on Intel or neon on ARM processors and so on and this actually makes a big difference because if you're running on floats you can run on eight floats at a time instead of one float with SSE - I believe and on two doubles instead of one double so you can get major performance improvements so basically in this case because breeze that the creator of breeze David Hall took the time to integrate this with Blas you know you're actually running linear algebra and Fortran even though you're writing Scala code which is really awesome for performance reasons and let's see what else I I did some plotting here this is actually the only side affecting code because plotting a side affecting this is using wisp 2 to do plotting on the web wisp comes with an unfiltered web server so you don't even have to start your web server here which is pretty neat and for linear regression specifically you have to implement the gradients and the cost function so the cost function is trivial sum of sum of squared residuals the gradient is equally simple and since this is breeze we can do a really really nice nice computation so dot T is transpose of the matrix and and we can just use you know operator overloading here : plus is the Ellyn element wise this of two vectors so again you know the fact that Scala supports overloading in this particular case is really useful because you can become in fix and you can use actual mathematical notation and let me just do a quick demo here so I generated this fake data set which has a Gaussian a bunch of gaussian random numbers for the feature and then target is that the intercept of 3 plus 10 times the the Gaussian from the from the feature plus plus another piece of Gaussian noise at the end and and as you can see the expected coefficients were 3 and 10 for as GDI got three point four and nine point two because as GD tends to converge very slowly even with momentum out I grab converged much faster and in fact it got the right result already but the interesting thing for example is that ogród was extremely unstable at the beginning well as GD had very small errors but as time went by I regret actually 1 and this is actually why I really like plotting stuff because you know if I look at 10,000 iterations and I have to look at all these numbers I don't really know what's going on but here I can actually see this and this is another reason to use notebooks as well which which have very good plotting and and that's basically it so you know linear regression might be a really trivial algorithm but what if you wanted to implement say LD a like Alex or I don't know support vector machines or let's say long short term memory neural networks right as long as you can distribute the optimizer and then collect the partial results from the stochastic computation I get your get your partial gradients and and be able to do the updates it's it should be simple to distribute really any algorithm that is amenable to distribution obviously not every algorithm is amenable distribution you know kind of really distribute linear programming and as VMS are actually really hard to scale by the way so there's one there's an implementation and they'll live but it's not but ma SVM is in general not very scalable but tons of algorithms are and since you can have this nice indirection of a type class between scale collections and rdd's and even flink datasets you should just write your code once and be able to scale it as long as your computation can be made stochastic so so that's it I guess the key thing is you know the Python ecosystem didn't have didn't have the libraries that they do right now a few years ago when the our ecosystem was essentially complete and people just hated using R and SAS and other technologies and they actually got busy to to build stuff my claim is that Scala may not be Python and may not you may not have Theano and PI brain and you know pandas and all of these things but we're not starting from scratch either there's already a really nice ecosystem we just need to keep building stuff so so I guess let me just bring up the slides for a second so this code is on github and this is actually part of the book that I'm writing with malcolm's so the bigger code which will which which will cover other algorithms that are it that we mentioned the book are in the second repo and we don't really have a date from manning when the Miep is gonna come out but I assume it's really soon since we have half the book written so hopefully the publisher will get busy soon and and and do their part but but we're covering stuff such as you know how to do a good online variance calculation and how to do naivebayes at scale and as VMS and locality-sensitive hashing and stuff like that so essentially the usual vanilla you know popular machine learning but the point of the book is not to show you just recipes its how use how you distribute those kinds of computations so if you get if you get stuck and you have to implement an algorithm that's not covered we're trying to you know promote the mindset of how to solve these problems not just provide a recipe book so that's sort of the intention and I guess that's it sorry spark using Python well first of all here's the problem with spark using Python right the spark I spark exists of course and you can use pandas and you can use you know everything from from by spark the problem is that you still have to distribute there your algorithms my current understanding of Pi spark is that the algorithm coverage in pi spark is much less than in regular ml lid and this is because you know these bindings have to be written so you don't have to rewrite the algorithm in Python to run Emma Lib but you have to do that Java bindings to the Python code base in order to expose those api's for the Python user so the Python coverage of algorithm is actually behind the Java and Scala coverage besides there's more to the machine learning ecosystem on the JVM than just that because as much as I used to poopoo you know Apache mahute a few years ago now they have a really nice Scala DSL for linear algebra and they're adding all of the algorithms they're adding right now are based on spark and they're actually trying to achieve parity with flink as well so there's quite a bit of work they're going at the moment so it's not the same hoot that I remember from say a year or two years ago and so I think building stuff in Python is relatively straightforward because you have a nice ecosystem already but if you have to distribute your computation by using for example none by then you might as well distributed using breeze and scour instead if since you have to make an effort to actually distribute the computation to begin with the same problem is the case with spark our although I think the spark are parody with Scala is going to be a little bit faster they've kind of accelerated to suffered but again it's parody it's not an advantage so there is no well that is actually quite urgent yeah what is what would be the why would it be important to have a symbolic tool books for Scala and yeah basically what why would that be important right so well first of all I'm not talking about this kind of uses that we know from Mathematica or maple where we you know provide a huge equation I want to just simplify it or solve a differential equation or whatever there are many other use cases for example siano for for deep learning has this feature of automatic differentiation and you can basically say oh I I just dreamed up a crazy architecture where you know one layer is gonna become volution 'el and other layer is gonna be a traditional perceptron layer and other layer is gonna be subsampling another gonna be is gonna be up sampling and other good is gonna be an inverse convolution as long as you know that that each particular layer you know how to construct any particular layer Tianna will figure out how to how to combine them and how to apply the chain rule in any abstract way for all the architectures that exist so you can compose a new neural network that never existed and it's just gonna do it and that is actually a big advantage there are many others but but this is actually the the one that's closest to my interest I haven't done like traditional symbolic computing map pub-style or a Mathematica style in a while but this is actually quite useful for certain problems yep cool cooled that that's really good news yeah so there are there are two libraries for symbolic computing one was Oscar and one was what Figaro yeah yeah actually heard about figural I never used it really there were also some really well-known Java libraries for quite a while although my understanding is that those particular libraries have a commercial license even though like academic uses free but the scalawags I the figure was definitely like Apache or something right or MIT yeah so so that's actually really good news I'm gonna check it out for sure thank you and the other questions or ideas all right all right thanks guys [Applause]