BDSBTB 2015: Marek Kolodziej, Unsupervised NLP Using Word Embeddings, Scala and Apache Spark
all right so this talk is gonna be about Skyland machine learning and I'm sure a lot of you are already doing machine learning in some shape or form with Emma lib or h2o or some other technology but I would just like to say why Scala first of all considering that there's so many other technologies out there like Python or R and so on and also to show how easy it is to build an application from scratch even if a library is non-existent so without further ado let's begin I'm just gonna briefly first of all refer to the existing machine learning and data science ecosystem in R in Python just for reference just to see what we might be missing in Scala or might not be missing necessarily so I'll try to look at pros and cons review the JVM alternatives not just Scala but predominantly Scala because of the syntactic sugar and and beauty that Scala provides us and and then I'll actually show a small demo what if for example you didn't have let's say linear regression of course you do have linear regression in in SPARC and I'm a Lib but just to show you how easy it is to build something and in fact if you look at the key problem with running linear regression at scale it's really the optimizer right if you wanted to do the same thing with neural networks as long as you can build an optimizer welt' and the rest scales right and and how did the vectorization vectorization is a perennial problem no matter what algorithm so again a lot of these learnings actually do move for move on to other algorithms well obviously I don't have to convince you of the fact that machine learning as ubiquitous from fraud detection and spam filtering which have been around for decades to recommender systems to OCR and even self-driving cars so it becomes more common even for non machine learning people to start dabbling in this area and because we have so many different use cases for machine learning natural language processing machine vision etc there actually tons of models you know people are fixated on this let's say deep learning these days but in fact there are tons and tons of algorithms that are very specialized even for something like regression right you can have linear regression low as Mars K and then decision tree regression random forest regression SVM neural networks etc right so much classification you know you could have naive Bayes decision trees random forests as VMs etc you can even have neural networks for cloud F for clustering such as self-organizing maps so obviously there are plenty of algorithms and some of them are actually really fine to you into very specific problems which is why it's nice thing about diversity of algorithms it's not just just fixate on one particular approach and you know when you have so many algorithms obviously doesn't want to code every single one from scratch it's it's quite a nightmare to to deal with this and if you look at say at the art ecosystem you see plenty of libraries in fact the the growth into our libraries has been exponential over the last couple of years and we may not have five thousand libraries for machine learning statistics and NLP in in Scala but we I'm gonna argue that we probably have enough to get started and to actually grow this this portfolio of course Python is nowhere near our when it comes to the algorithmic coverage but scikit-learn is definitely a very good started machine learning and so is an LT k for for language processing and arguably in some areas Python is actually excelling like theano is a really great library for deep learning as is pi brain and this thing we can say probably about torch although that's Lua so that's another exotic solution but you definitely have a good MIT of linear algebra and numpy and data frame processing and founders etc so definitely a rich ecosystem same with plotting in math populate how just to show how easy it is to write something in Python and just to basically make the point that this is the kind of stuff we really should be aiming for in scale and not just you know fixate on type in friends and you know implicit conversions and and you know monads and whatnot you know what as a data scientist actually want to be able to write something like this in 30 lines of code build an OCR system at least 400 handwritten digit recognition actually out of this entire code base only two lines or three lines of code are used for training the the support vector machine and to predict the rest is basically for printing out the confusion matrices and doing visualization so in fact training and prediction is basically three lines of code and this is the kind of stuff we would really expect from from a modern machine learning library but of course you know despite all the benefits that our provides in same with Python you have lots of problems right first of all dynamic typing we were at a Scala conference so I assume that's kind of a given that we would prefer static typing not just because of the potential type mismatches I mean I'm sure if to those of you who've been using our are especially Python for for machine learning it must have happened that you were writing a print statement and you concatenated a an integer with a string and by in Python blue ops saying Oh explicit is better than implicit so you should have done an STR on that integer right I don't want to know that things at runtime I want to know them at compile time I don't want to waste my time running a simulation for a few hours and then having him blow up right but they're you know even more fundamental problems like like typos right I can just miss label the the the method name and the code blows up at some point and I don't know about this to it so it's waste a week we're running a deep learning algorithm or something right so clearly clearly it would be nice to have static typing but at the same time I wouldn't probably give up static typing if I hadn't no no data science ecosystem at all because people expect me to deliver results right but you know just going back to the problems with with our in Python these languages definitely don't have good runtimes I would argue that the JVM is also not a great runtime and I'll tell you why specifically for for machine learning and and numeric processing I mean one obvious case is boxing another case is cash locality and and vectorization so the JVM has its own problems but if I were to compare this to say R or Python they definitely have more problems are does everything by copy so so basically you have a memory problem very quickly Python has the global interpreter lock which is kind of sad in the era when we have multi-core cell phones let alone servers that you have to do everything through multi processing like it's I don't know developing web apps circa 1995 you know when people were doing CGI you know it's kind of the same the same problem essentially and it's kind of too late to be dealing with multi processing when when when multi-core is the way even the Raspberry Pi works with 4 cores these days right and execution is actually slow - so even if use something like PI Pi which is a just-in-time compiler for Python or the ardh bytecode compiler they're still relatively so compared to the JVM so even for a single threaded processor processing they're slow as many machine learning people say you know R is fast when it's written in C++ and same with Python you know a lot of these libraries have been written in C++ but that means you have to juggle two languages so that's kind of a pain and of course when you think about in about math specifically math is declarative it's not imperative right i what does it mean in math I take a logarithm of iPod and do some I plus plus in the in the middle of the of this code right I'm not side effecting anything I'm just pushing in some value and expecting a result out of the function right a function can have a limited domain right it can be a partial function which Scala actually supports but I'm not side effecting math is always referentially transparent it's always item potent and some things are explicitly explicitly declarative and it's not just beyond Oh numbers and factorials and Fibonacci numbers but it's for example defining how the activation of a neural network works right the the the input to the to the layer is actually the output of a previous layer so you can recurse through the whole structure declaratively so math is just very amenable to this kind of thinking and you know people say r is a functional programming language right because you can have a function within a function and the function can be a first-class citizen so it's kind of functional right but it doesn't even have tail recursion it blows up here in this particular example right and and then Scala no problem right because it was designed to be really functional so okay we know that there are problems with R and by phone and we know that ska could be potentially the solution but how does the ecosystem look right if I need to write a singular value decomposition before I do any math right and spend time writing writing the Scala code for it that's not good right so I so we have to we have to have some sense of what's out there so that we don't have to start from scratch so first of all numeric abstractions there's the actual library by Adam Bingle I'm not sure if Adam is here but I stuck him earlier he really did a great job with this library there aspire by Erik assignment thumb Switzer really phenomenal numeric abstractions including floating point filters that are almost as accurate as a big decimal but er fast better as fast as a float these guys are really brilliant and also really beautiful dsls and macros and whatnot these guys you know or with indefinitely Scala whizzes when it comes to linear algebra which is sort of the bread and butter of machine learning there are plenty of Java libraries like eg ml la forge and4 J is a java-based library although now it has a scale wrapper there's nd for s that's Adam Gibson's library which was designed as part of the deploring 4j project and my personal favorite is breeze which is part of the scott scott NLP project led by David Hall this particular library is not just a beautiful Scout the SL which supports the vast majority of linear algebra that he would ever want essentially at the MATLAB level but it also actually calls Fortran native Blas code to do the numeric processing and this is actually a really important point as I mentioned the JVM is kind of socks at numerix and to make a point it barely supports any kind of vectorization Java 7 started supporting basic vectorization for loops but it doesn't support MMX properly it doesn't support SSE doesn't support AV x doesn't and and vectorization has been around for for Intel CPUs since like 1995 so if you can process eight floating-point numbers on a single core instead of one at any given time that's a big boost right so clearly it's important to be able to run vectorized code and unfortunately for in the java world linear algebra is fast if it's written in Fortran but these guys actually did the J&I bindings to Blas so we don't have to worry about this while writing beautiful scalloped code you know that the Python community is definitely passionate about reproducible research and notebook style computing so Andy Petrella who was myself in here he he created the SPARC notebook which is an extension of the original Scala notebook which was about a port of the ipython notebook to Scala Jupiter Scala also has a Scala kernel that's based on the Jupiter Python project there's Apache Zeppelin there's of course the commercial data break solution as well which supports Scala but also Python and sequel so so we can do reproducible research in notebooks as well for plotting you know we don't quite have a new plot or math lab plotting or say my log lib but I would say wisp is a good first start there's the Lightning visualization server which is actually language agnostic it just has rest api's but there's a scholar wrapper for it breeze has somebody sick visualization I guess this is the weakest polling but you can actually work with what you have already and for machine learning itself breeze offers a bunch of algorithms there's a melody for spark if you want to run in the distributed setting h2o has has its own spark port called sparkling water which runs off heap and is very efficient and they have some of their own algorithms there's deep learning for J which does some of the deep learning models as well so so we have some existing ecosystem not quite the Python one or but but a good start right good enough to actually get productive but my favorite part of the JVM ecosystem is actually the Nobuko system because this one actually beats NLT k the parsing algorithms and part of speech tagging algorithms and so on are actually much better than the noun and LT K so so even scientifically it outperforms the Python ecosystem and definitely in terms of the speed and and and and feature set so here we have Stanford Cornell P and link pipe and Mawlid these are Java based libraries but epic Park and factory the three ones at the top are actually fully Scala based and in fact parked the that the middle icon at the top is a GPU based parser so you can even get the GPU speed for for parsing so so that the NLP ecosystem is definitely really good on the JVM even if you have problems with with some other areas a little bit but what if there's no library for something right how difficult is it to build it well I would argue that it's not because scholars a good language so you can get productive pretty quickly even if you have to build something from scratch so why well first of all function is our first citizens so if you're writing an optimizer you can just support provide the gradient and and the cost function as an argument right no need to write like new rapper involve like in Java or something like that right you have functors in monads so you can do whole pipelines of transformations which is really convenient and of course the the the functional programming practices encourage referential transparency immutability lack of side-effects declarative programming and other things that are actually really useful for math and they're really useful for writing bug-free code so that definitely helps but syntactically Scala supports a lot of things that are extremely convenient such as implicit conversions and as I'm gonna show in a demo in a second you can for example build one model to run on both Scala collections and rdd's through implicit and this is actually really convenient Dana Brooks was talking at the spark summit how they now want to have a version of ml lib that runs on a single machine not just on spark because when you're running on a single machine you actually get a lot of overhead from spark such as right having the scheduler execute and message passing and a lot of things that actually are not necessary in process and so they actually want to have a version of ml Lib that that that runs in process on the Scala collections but if you have implicit conversions you can just write your code in terms of an abstraction in terms of some trades right which will redirect either disk our collections or rdd's without having to rewrite any code and if you write another implementation of this trade you can run on flink or something right so so employees are actually quite powerful and of course things to type inference named function arguments you don't have to write the Builder pattern like in Java here you don't have to remember the order of the arguments of the constructor and stuff like that and write final boilerplate boilerplate right so a big time-saver so let's actually build something I'm gonna argue that in 15 minutes we can build two after my optimization algorithms to cast a gradient descent and our using breezes linear algebra so this code will actually run vectorized in fortran at blazing speed and then we're gonna write linear regression cost and gradient functions the implicit and I'm gonna show the implicit conversions to abstract the computation over Scala collections and spark so you just provide whatever datatype you want and it's gonna run on either and then plant the applaud the optimization results to show how how the algorithm converged and run against the synthetic data sets to validate the results so first of all just to quickly review of linear regression for those who may not remember you have an explanatory variable or a so-called feature and you have a dependent variable or a so-called target you can have multiple features predicting a target but here we just have one feature X which predicts y and and that we have some error term associated with the with the this XY pair and we're trying to fit the the best line that minimizes the sum of squared errors so you know we could write it as as a vectorized equation and and and these matrices below actually show show you how how to write it in turn in terms of vectors so here's your actual vectorized representation which is how we repair we're going to exploit Sindhi operations on a cpu through through blas and stochastic gradient descent updates your weights according to a learning rate but you you provide a learning rate of kneeling so that the algorithm stabilizes over time you can do a square root or just n times i where n is the number of observations for per iteration and i is the number of the iteration i regret actually scales by the gradient so in the direction and when it's the gradient sorry on that the learning is very unstable and the gradients are really large over time the learning rate is attenuated more than in the direction in which the gradients are small and the convergence is stable so this is kind of like DF IDF for learning for for optimization as the original authors of the other grad paper point put it and the cost is just the sum of squared residuals X beta minus y and since we're doing linear algebra we're not doing summation we're just doing transpose x do it the exhibit a minus y and the gradient is just the partial derivative like so okay so let's just briefly look at this and at the plantation by the way this code is on github and I have a separate repo called Scout data science but this talk was inspired by the work that my colleague nitro Malcolm and myself are doing for a book that we're writing formatting press it's going to be MEAP in a couple of weeks hopefully so so be on the lookout for for the actual MEAP release but but the github repo already exists and and let me just show you some code I guess the screen is kind of low res but I'll do what I can so let's see okay so first of all I have to have some representation of your data it could be just a regular being like like this with a target for for this regression model which is a double because it's a continuous value and then features which is a dense vector of double and this dense vector is actually a brief type which which if you have your Blas routines installed and you important breeze natives it actually redirects to Blas so sorry computations are actually running natively not on the JVM then you have to have some optimization history with sequence of caused values sequence of weight values if you're doing early stopping and want to backtrack to the best value before before the code starts to overfit and you have your gradients which which allow you to scale the the learning rate for autograph then we have a bunch of types for the grad function cost function weight update with initialization so these are essentially just function objects and I could have written them without without these case classes but the reason I wanted case classes as opposed to just lam does is because if you have a lot of named arguments you would like to know you would like to be able to refer them by name keyword args essentially as opposed to remember remembering their position if you have three double arguments how are you going to remember the position so if you actually have a case class it's gonna allow you to call by name let's see and then we have this vectorized datatype which explicitly takes care of the fact that instead of taking one target value at a time which is a double you actually take a vector of doubles and the features are not the vector of doubles they're our matrix because each row is one example and and n rows give you an examples at at times are doing a vectorized operation let's see after this we have to make sure that we can basically repre okay let me first show you that this data so this data is actually due to Malcolm he he wrote this for for the book and this is what I was talking about the possibility of having a trade with standard methods such as Map Reduce take etc and and instead of using say a scale collection or an RDD you use this this trade but the implicit conversions generate instances that delegate to scale collections are rdd's so then if you provide a data an object that's an RDD and it'll run on spark and if you provide an object that's a scale collection it's going to run locally without any of the spark overhead and for your implicit conversions do traversable which is which is an underlying type in the scale collections library or an implicit conversion to to the RTD and here's your concrete implementation for for traversable x' and another implementation for rdd's and this is really useful you could provide another implementation for flink and run on flink instead of spark so so this is really where scala shines with the implicit conversions and now we probably should look at the optimizer so the optimizer you know it takes the standard things for for stochastic algorithms such as the initial learning rate the momentum that and and here it actually takes the gradient function cost function so you provide first-class functions right for for a particular algorithm whether it's logistic regression or linear regression or even a neural network or something and and the update function is actually the specific implementation of the update rule so stochastic gradient descent is gonna have a different update rule than an autograph but again it's a first-class function so no boilerplate there and many matching because we're doing this stochastic convergence and we provide the data and after that it's essentially a fault we basically applied this this this update function and number of times where n is the number of iterations and and that's it so this is essentially just iteration without writing for loops or something and now we just need the concrete implementation of the of the optimizers so stochastic gradient descent essentially takes the learning rate divides it by this annealing step which is the sample size in the iteration number you can have momentum which is basically a multiplier for the previous weight update so you attenuate the gradients in the directions in which they're unstable and and increase them in the directions in which they're stable so if you have a narrow valley it actually propels it against this narrow narrow valley and ignores the the the seesaws to the sides that's what the momentum does essentially and and let me just update the weights and you have the momentum if you choose to do that and you provide this up to history object so that's all of as GD and because we're using this data trade not the Scala Rd at the RDD or the skull collection concrete type this will run on both spark and and in Scala collections and also because scale collections don't have the aggregate method which has a sequential step and and and and parallel step you could basically this straight can have the redirection of the of the of the aggregate method in the case of a scale collection to essentially a fold whereas in in the case of the spark RDD it actually redirects to the to the aggregate method which already exists in the spark and for autograph the scale by the diagonal of the Hessian which we do here and again it's very little code because we were using breeze which which benefits from the fact that Scott has operator overloading so we don't have to write delta ad when can do plus right or we're doing a Hadamard product we can just say : star instead of calling dog Hadamard like we would have to do in Java or something so it basically looks very much like math which is nice because then we can do a one-to-one comparison between the actual mathematical formula and the code so that's really the testament to why sometimes operator overloading is useful and white kind of sucks that Java dropped it from C++ and Scott has it back because when you're doing math and you know what these symbols are it may actually be useful and let's see and then I do some plotting which is going which is probably the only imperative part of the code because plotting aside affecting after all and this is actually using wisp so it starts a web server that draws some highcharts plots and let's see oh and we just need the linear regression cost function and gradient so the cost function is really simple as just the sum of squared residuals so we have the features time waits mine as a target and and we just map over these values square them and do a reduce left right it's really simple since it's just linear regression and for gradients it's equally simple right we just to feature some weights - target times features transpose just like on this slide so again I'm just really following the the math and just and just typing it up without actually having to think about how the API will look like so it's really nice because it's very close to the original math but that's thanks to David Hills breeze library for linear algebra and now for this ad I grabbed mo I'm actually creating a fake data set which basically has an intercept of 3 and a slope of 10 and I'm generating a thousand of these random very 'it's that are distributed as a standard normal and and so the targets are the intercept plus the slope x times the the normal normal variate from from the features and then i'm just trying to see if when when I when I do this convergence if I actually get the coefficients three and ten or close to that value and as you can see because of this nice this data wrapper that Malcolm wrote I can actually create either this data for an RDD or for our local Scala collections and that I call optimize linear regression cost and linear regression gradients are just functions thanks to scholars FB features and that's it and now let's just run it and I actually have a wisp lot so I'll be able to show you something at the end it run it runs a bunch of iterations it's probably way too many relative to what I need so it may take a little bit of time but here's the pot and the expected coefficients were three and ten and for example stochastic gradient descent got three point four and nine point two but a Degrelle got three and ten so I regret definitely converged better and as you can see it may have been unstable at the beginning whereas the gasps degree in this end which is in light blue was much more stable the gust was much less but if you look towards the end it actually converged to a much lower value so I regret it was unstable but it actually converged a better value add yet so one nice thing that you know python and matlab and and our people take for granted is the plotting you know you may look at thousands of numbers but you can quickly figure out what the heck is going on if you if you're able to visualize it and even in scala you can use something like wish or other plotting libraries to to basically make some progress here but I guess the point is here you know linear regression is something that already exists in that moment but what if I wanted it a you know a deep belief network or you know some particular rendition of support vector machine that didn't exist in the lib or in or in some non distributed sky library or what if I wanted you know the latest to implement the latest paper like let's say I'm in the word Tyvek era and and the glove paper by by Stanford is really East and I want to implement it well I guess I was trying to just make the point that if if something doesn't exist because Scala has nice functional programming features and because you already have linear algebra primitives and other dsls to get you started even you even have plotting you don't have to completely start from scratch and you can actually build something in an afternoon maybe not a super complex model but but you don't have to essentially write you know a lot of C++ Malick's and stuff to blog to Python later so I would say I would say Scala is definitely nice environment even if you're forced to build something from from scratch because it may not already exist in the ecosystem the way it would say an R and it and it definitely will scale way way better than an R which will probably you know throw up and and throw some garbage collection error because because you loved it too much data so I guess I guess that the closing point here is you know the Python ecosystem that didn't exist for machine learning when the our ecosystem was already built out and the Python people realized you know what we don't want to use R we want to use Python and let's build something and and Scala ecosystem and the JVM ecosystem in general already exists due to some degree to a reasonable degree I would argue especially for natural language processing and if something doesn't exist then as a community we might as well just build it so because we already have a good start so I guess that's it any questions yep oh the MEAP that depends on the publisher we have half of the book written but they want some small like stylistic Corrections and I think in the next maybe six weeks or something we will we will make them but at that point it will really depend on whether the publisher decides to release it or not but but that's essentially the timeline hopefully Manning yes yep well the current title is machine learning for big data and the underlying technology that we're talking about a spark but you know until they actually release it who knows what the final version will be I guess that's the current current title but that's a good question so any other questions yep for blotting yeah aside from blotting is there anything else missing well I would say a bunch of things first of all symbolic processing this is sort of a given in Python with simp I and in MATLAB and in octave but I would say are kind of misses it too so it's not completely bad but it's nice to be able to say I have this humongous equation and so take the derivative of it without me having to take a derivative so that's kind of cool the other thing is definitely deep learning you know deep learning for jig is a good start and I'm gonna ignore the fact that it's Java because they are working in a scholar wrapper the bigger problem is that their focus more on image processing right now than the natural language processing and we at nitro are focused on NLP so that is an issue but honestly I would say I would say definitely the biggest issue is blogging to me at least because a lot of other stuff exists and also there's kind of there's one issue that sort of is is difficult with static typing which is that a lot of what for example theano does is basically meta programming and when you have a language that it's dynamic and can write itself at runtime and execute itself you know like when you look at for example the Ruby and rails community like probably half of their code is meta programming and I'm trying to debug it and they're like oh this code hasn't been written yet because it writes itself at runtime and sure it's really horrible to debug but at the same time it's easier to do something like automatic differentiation where you can imply say a symbolic toolbox and write the derivatives at runtime based on whatever the user decided to to concoct as the model and this is definitely harder for static languages but I think it's definitely worthy goal maybe one of solution is to basically say we have a subset of models that have certain architectures and we allow some flexibility but the types are generally known and it's just the composition that becomes flexible and at that point you don't have to write arbitrary derivatives at runtime or something so this is definitely one of the things that that probably still have to be done but but I would say the start is definitely ok you know the just having the linear algebra is actually powerful when I sit in MATLAB most of what I do is just a matrix operations and plotting and and I would say you know that's that's what already exists and also I would say they're all there are a lot of good Java bindings for that you can reuse in Scala such as for example Java CV if you want to use the C++ base the opencv library for machine vision so sure it's Java but you can always write some nicer abstraction on top of it in Scala but you already have the JVM binding so you don't have to write J&I code or something so it's I would say it's a good start you know so sorry what's my best recommendation for what yeah I would say there's definitely deep learning for Jay but as I mentioned it really depends on what you're trying to do if you want to do say convolutional neural networks because you're doing machine vision and that's good enough if you want to do let's say long short term memory networks or recurrent neural networks for for natural language processing then it's probably not there yet but there are other libraries for Scala such as neuron which are sort of more NLP focused it's neither of these solutions probably approaches to Theano but there are also sort of more traditional neural networks libraries such as n cog which are Java based and they're actually really really solid they have open CL bindings they have a lot of architectures that people actually ignored in deep learning such as self-organizing maps which are extremely good non-linear clustering algorithms so there's a variety basically if you want to look at more traditional neural networks and cog is a great solution if it's more NLP focus then it's probably neuron if it's more vision focus it's deep learning for J these things are definitely evolving still well I think FB is just a good design principle in general for math because math is declarative and math is side-effect free and and a lot of the abstractions are based on the ability to pass functions around so if you have an optimizer and you want to say dude this is my weight update you know the weight update will be probably a function that takes some input and produces some output so if you can provide the function as an argument that's actually cool I mean that's not even possible in Java eight but but that's exactly the point that Java it is now catching up with Scout because they realized that that actually having lambdas as first-class citizens instead of as some you know anonymous classes with within method inside is actually a useful idea also the also Java doesn't have tail recursion which is a problem if you're trying to replace looping with a recursion in many cases it's not just just because I want to be religiously functional but because a recursive representation of the math actually makes sense to the mathematician so so so I guess I guess functional programming is pretty useful I mean even even if you look at spark for examples PI spark is following the same principles and Python at least has lambdas it doesn't have advanced functional programming but it's at least Islam doesn't that that seems to be useful type inference helps with reducing boilerplate since this is a statically typed language so I would say that general design is good of Scala for these kinds of problems I mean the biggest problem as I said is the JVM itself right the JVM will lift your you know primitives into in two instances of capital double or capital integer in the list when you wanted primitive ends and it's gonna destroy your cache locality and generate garbage collection and and Java will not vectorize properly the only two languages that do any kind of victory zation I think are well three languages C C++ and for Tron essentially Java started in Java seven but it's not nearly where it should be there's some talk about Java then actually having heterogeneous computing so like automatic GPU computing and stuff so what Martin was saying that the effort is being done for Scala there might be a general effort for the JVM as it interprets the bytecode so maybe maybe that's a duplicated effort because the JVM will have it on its own but Java has been kind of behind schedule with a lot of their promises that are already listed on the on the community proposals so maybe it's better if sky actually leads the way but these are the big problems you know the JVM is still 100 times faster than Python for machine learning but it's definitely slower than C++ because of the memory locality cache locality you know garbage collection boxing and all kinds of other junk that really shouldn't be there java 9 is gonna have value classes so at least you won't have boxing anymore if you want to have a tuple that people will not be boxed anymore and I think that's a huge win but there are other things such as vectorization which may not be there in java 9 yet so which is why we actually call Fortran at the end of the day but at least from Scala so you'd at least see it on the right Fortran anymore you just call it a nice Scala DSL for it so anything else all right I guess that's it for me thank you