Scale By The Bay 2020: Shadaj Laddad, Data Science in Scala with ScalaPy

Scale By The Bay 2020: Shadaj Laddad, Data Science in Scala with ScalaPy

Recording: Scale By The Bay 2020: Shadaj Laddad, Data Science in Scala with ScalaPy

[Music] uh hi everyone i'm shidaj and today i'm really excited to be talking about data science in scala with scala pi um so let's just dive right in why scala pi so the core idea is that today data science has a split ecosystem we see scala being used in a lot of production environments for handling large-scale data processing and those types of production applications while python is really useful for experimentation and you see things like jupiter notebooks which have kind of you've developed this ecosystem around python for uh doing quick experiments and exploring data and we also see this in the types of libraries that are available on both ends apparently there's no audio there might be audio okay apparently it should be fixed um but we we see spark which is a built-in scala for handling large data sets whereas numpy has this really rich set of apis for manipulating matrices using linear algebra algorithms and so we kind of see this split where skull is really good for kind of these production large-scale uses whereas python is better for experimentation and there's no real kind of crossover here there's no there are not many libraries in skull for experimentation and not equally not many libraries in python for these types of large-scale data science and so us as developers and data scientists it's we often have to perform this context switch between these two languages so you might start out in python do some exploration of some machine learning models and then you have to switch to scala for actually productionizing that and that's a really expensive context switch so what how can we bring python and scala together was the question that kind of started uh the exploration of uh scala pi we want we have these two ecosystems we want to use them together how can we bring them together and that's where skull pi comes in so scala pi lets you use python libraries from scala code it lets you do this both in jvm and native code so you can run it regularly with scala on the jvm but you can also compile your code ahead of time with skull native you get a seamless interoperability layer which means that any type of scala value can be shared to python any type of python value can be brought into scala without any trouble and you can use python values from your scholar code as if they were just regular scala objects and lastly you can get a fully uh type safe interface so uh if especially in our skull code we're really focused on having types of code and that's really helpful to catch bugs before they happen in production but python is a dynamically typed language so scala pi gives you the ability to create type facades which lets you add static types on top of python libraries so that when you're interacting them interacting with them from scala everything is type safe so before we jump into the internals let's write some code um so uh skullify works with jupiter notebooks and you can use them through the almond implementation of the scala kernel for jupiter here i've actually loaded it directly into google cola which is a hosted jupiter environment so we're going to go through two examples so first we're just going to do some simple kind of linear regression using tensorflow and in the second we'll build up a full deep learning model so to get started let's load uh skull pie okay i need to approve this so we get some nice utilities here provided by almond that let us import libraries just using a regular skull import statement and then i can load up scala pi so we see that we've loaded up skull pi and then we can just start talking to python libraries so let's say we want to use numpy in python numpy is a module and so in scala pi if i want to load that into my code i just use the pi dot module api which imports a module and it exposes it as just a regular scala value this is a vowel so just regular scott code here we're also going to use tensorflow in this case we're loading the tensorflow 1 compatibility module uh since i haven't been able to port this example over to tensorflow 2.0 just yet uh so we're going to be using the tensorflow 1 api so we've loaded tensorflow we've loaded numpy now let's create some training data so we're just gonna do some simple linear regression so uh first we want to generate a hundred random numbers for our uh x uh points so now we can just use numpy um we can call api methods just completely regularly as if it was a scholar api um but you'll notice that this code is actually identical to what you would write in python one of the key goals of skull pie with that seamless intro player is that you should be able to take any examples you see from the python world and put port them to scala without any trouble now in this case our apis are going to be dynamically typed because we just loaded a module so skullply doesn't know what specific static types they should have so that makes it really easy to get started using your favorite libraries without having to define your static types but we'll see later in this talk how we can actually then once we want to productionize this code install some static type definitions that let us make sure that we're interacting with these libraries safely scalable also handles all the nice python goodies like operator overloading so here we can scale this vector of a bunch of integers add something and so here we're just setting the uh y-intercept and slope for training data and so what our goal is going to be now is setting up kind of a tensorflow graph which will try to infer the y-intercept and slope of these points just by looking at this training data so you see we have our training data generated again it's all just regular scala values next we initialize some tensorflow values so now things get a bit more interesting so we want to define a tensorflow variable but we want to kind of initialize it as a random value that's uniform from negative one to one now tensorflow requires us to give it a python list which describes the dimensions of this tensor that we're going to be initializing and so here we can take a scholar sequence this is a regular skull sequence and just convert it into a python sequence instead by using this two python proxy we'll see why we have this special proxy method later similarly we can initialize of zero so we can have a one-dimensional vector of zeros for our intercept and then we just define our graph by saying that our predictions should always be our input x data multiplied by our slope and we add our y intercept all right so now we've initialized that then we just have a couple steps remaining this is just like the type of code you'd write in python but we're doing this all from scala and so we can use whatever scholar goodies we want as well so we're going to initialize our optimizer and lastly here now we can mix a bit of skull and python together so we're going to do a loop of 200 iterations in each one we'll run an iteration of training and at every 20th step we'll print out the current uh slope and y-intercept that we've predicted and again we can use string interpolation things like that and mix and match these scala and python values you'll notice here we're doing this thing called bracket access so we'll see some later that uh there are some syntactic features in uh python like using brackets to access elements from an array that don't have an equivalent in scala and spell we just use regular method method calls for loading values out of sequences and so if we want to kind of map those python syntactic features into skull we offer custom apis so in this case we use bracket access to perform that type of bracketing operation and we've run it and we see that we are able to predict our slope and y-intercept and this was just using all tensorflow under the hood but now to see a bit more of an interesting example i'll go through this a bit faster uh since uh uh uh we uh we then we can jump into how skeletal actually works under the hood but what's really interesting here is tensorflow supports gpus google collab offers us gpu back-ends to run our machine learning operations on and skalpid doesn't limit how you can use your python libraries so if if we want to use tensorflow with our gpu back-end we can do that with no trouble at all scholarly supports any python library you can throw at it so we'll get started again with this a similar type of initialization here we're going to be doing a bit more of a fancier uh a bit more fancy code uh to initi to set up a hook that will intercept python's print requests and pipe them through the jvm instead just so that we can see the logs that tensorflow prints out then we import numpy and tensorflow we load the mnist dataset so these are pictures of handwriting so we're going to be recognizing handwriting as we loaded some data we're going to use the keras api provided by python again this is very familiar stuff but we see things like here for example we can use a regular scala tuple pass it in we can have named parameters just like we have in python but here we're using scala's named parameters um and so we're able to initialize a bunch of convolutional layers um we can print out the summary so this is tensorflow showing us what our model looks like so far we can add some uh we can add our final layers to flatten out our our uh kind of two-dimensional uh tensors into just a flat tensor and then apply some dense layers uh print out our final model and then we can actually train and this would be super slow on a cpu but uh because we're uh running on collab and we have access to a gpu we'll see that we're able to run through these training steps super fast even though we're dealing with a huge data set of images um so we can let this run it'll be done in a couple seconds and then lastly we can just say okay i want to evaluate my model on my test set i know that tensorflow will give me back a list of doubles i can convert that into a scala sequence of doubles so here we see kind of the the transformation abilities where we can take python values convert them into scala and print out our test loss and test accuracy and so after we run through we see that we're uh getting an accuracy of 98.97 so that's really awesome so in just kind of a couple lines of code without really anything uh too out of the ordinary we're able to use the python apis just like regular call the apis that we see in the python docs but do everything from scala and that's what scala pi is all about so now that we've seen skull pine action let's talk a bit about how it works because that's where all the magic happens so to begin let's talk about how scala pi talks to python there are a lot of interrupt projects for scholar like scholar gs to compile skeleton javascript scala native to composite ahead of time to native code scalp has a bit different in how it achieves interop so the approach that scallop takes is it actually embeds a python interpreter inside your runtime environment so you load skull pi as just a regular library you just add it and that's how we were able to bring it into our jupiter environment without any special tricks scala pi will initialize a c python interpreter in either your jvm environment or scholarly depending on which back-end you're using and then all that changes between these two platforms is how scala pi talks to the interpreter so if you're on the jvm scalpie will use jna which is a nice kind of high level foreign function interface implementation and so it'll use that to talk to the c python interpreter and on skull native skull native provides us kind of a direct form function interface that that we can use and then all the magic kind of is shared across these uh these platforms other than this kind of how we actually specifically talk to c functions and deal with pointers everything is shared across the two platforms so for example in the c python interpreter if we have these two values like an array and maybe a reference to a module uh tensorflow then we can have in scala pi two pi values so a pi value is kind of an opaque reference to a python value so pi values hold a reference to the actual native pointer for this underlying python value but nothing's typed at this level all any reference to any python values just stored in this pi value now the reason we have this intermediate layer is a lot of the work that goes on in skull pi is for heap synchronization because since we're embedding the python interpreter we must have we have we have a whole heap for all the python objects as well as a heap for your skull objects and so we need to make sure that references across this boundary are all uh kept safe and we make sure that we're not using something after it's freed or keeping anything around longer than we need it so this pi value handles that level of making sure that all our references to python objects are safe but what you actually deal with when you're interacting with python objects in your end user code are these nice high level typed python python facades so you have things like pi.list which is representing a python list pi dot module for modules and and more and so you just deal with these types and skull pie handles of the interaction with the with the underlying python interpreter for you so to look a little bit into kind of the inner bits of how uh scalabi handles this we actually have a very lightweight kind of platform object and the goal this platform object is to provide the core c foreign function interface kind of apis we need for both the jna and scala native through a common interface so here we can use type aliases to say that on jna a pointer is just the jna pointer type whereas in skull native it's a pointer of bytes and we have additional types for pointer pointers um skull native uses zoned allocations and so we have this with zone which on skull native will actually create a zone on jna since we uh jna just uses kind of regular java garbage collection to determine when to free pieces of memory we don't need it so we just uh use a no op and then we have things like for example if i want to allocate a pointer to a pointer we have different apis for how to do that on gna versus scala native but we kind of wrap these both into a common api and so then we can share the entire logic for actually talking to specific python apis in addition scala native is our skull in general we we often have lots of threads we have really great support for concurrency and scholar through things like futures and so we want to make sure that you can bring python into your skull code without having to deal with uh thread safety issues so python has this thing called a global interpreter lock that means that whenever you kind of call into the the python interpreter you have to hold this lock and skull pi will handle that automatically for you so whenever you're interacting with python objects it'll make sure to grab the lock uh exactly when necessary um and hold it for the minimum amount of time to perform those interactions across the barrier with python so now that we've learned a little bit about how kind of scala pi deals with the python interpreter now we can talk about the higher level pieces how do we mix scala and python which have some similarities but a lot of differences uh how do we mix them seamlessly so when we were working in our jupiter notebooks uh we were able to get started really quickly and that's a result of scala pi's dynamically typed apis so by default whenever you import a module or if you're trying to load something from the global scope you get a dynamically typed api this is similar to scholar js's js.dynamics so we have a pi dot dynamic equivalent so the reason that you start out dynamic is you just want to get up and running with numpy or tensorflow or your favorite python library you don't want to deal with all the effort of having to define subject types especially if the library authors in the python end have not kind of declared specific types for those api methods in things like docs so you just want to get started really quickly and scalpie lets you do that so by using scala's uh dynamic type which makes it easy to introduce dynamic types into existing scala code you can interact with all of these objects as if they were dynamically typed values in scala so for example if you take a list and you call append since your list is a dynamic type this actually expands to uh by the scholar compiler to this applied dynamic call and so this means that you can quickly start accessing apis without having to create all your definitions you also get a feature in skull pie called what we call python interpolation so scala has string interpolation which is a really powerful feature and it's actually defined in kind of an abstract manner so we can define our own custom interpolators scalify has the pi quote interpolator which lets you define snippets of python code where you can inject variables from scala so for example i can define a list a scala variable and say i would like you to execute this code getting the sum and inject this list inside there now what scalp actually does under the hood is actually creates a temporary scope loads all of the things your all of the values you're injecting into that scope and then just executes uh that expression but this makes it really easy for example if you see a snippet of code on the web and you just want to get it running in your skull code real quick without even having to translate anything into the scala equivalence you can just run it in your scala code another thing that we need to kind of bridge between python and scala are specific syntax elements that exist in python but don't really have an equivalent in scala so for example one of these is the with uh keyword so with lets you say uh take a value and kind of temporarily use it in a scope and free it afterwards so this is a common pattern used for files where you say i want to open up the file uh use do some operations with it and then release it afterwards one of the really nice things in scala is because we have such nice language features like this currying syntax we can actually get a fairly close equivalent to the python syntax without having to modify the compiler or anything this is just regular apis provided by skull pi so for example we can do pi dot with we pass in the value we want to temporarily use as well as a function for what we want to do once we've kind of temporarily loaded that value up so for example we can get a file and grab its encoding another thing we see here is we're going to start taking a look at more static types you can take python values and cast them into scala types at any time by calling so that's what we're doing in this case where we get file.encoding which by default is a dynamically typed value but we say hey i know this is going to be a string just convert it into a scholar string for me now another really cool thing that scalable handles is a reverse interop so you so so far we've seen scala calling and calling python methods but we can also send scala code into python so that python can call back to skull so we achieve this by having a conversion between scholar functions and python lambdas so for example here if i define a function that takes an int and adds one to it i can actually convert it into a python lambda and use it for the map function call so python has a map api similar to kind of how skull has it um and so map takes the first parameter is a lambda which describes the transformation for every element and the second method a second parameter is the sequence of elements that you want to transform so here we can just say i would like you to map but using the skull function now skull pi will detect that you have a skull function but you're trying to call python method so it'll automatically introduce a conversion that wraps this into a python lambda so this actually requires a bit of kind of trickery under the hood to get this to work right because again the c python interpreter and the jvm or scala native have independent heaps and they have their own garbage collection schemes so you have to make sure that we're handling these references across the boundary safely so if i have a skull function that i want to send into python what i first do is register it with this reverse reference manager so this is kind of a map that sits in our scala code which keeps track of all of the references that python currently holds to scala values what we do is to represent this reference in the world of python we create what we call a tracker object so a tracker object kind of represents a reference to the scala function and it has a finalizer installed so that we know when python no longer needs the reference to that scala value and so we can clean things up appropriately then we create a general handler so this is so when you want to create a lambda in python you just have to pass it a c function pointer since we're we can only pass in c function pointers that means this must be a static function definition so we have a general handler that will be used for all lambdas but what the general handler will do is it'll take a look at the specific lambda instance it's being used for pull up the function associated with that uh lambda and then call that skull function under the hood and so with all these pieces in place we can just create a lambda that says my implementation is the general handler but i also want to associate this tracker with it so that when the genera general handler is called it can look at the tracker pull up the skull function associated with that tracker and execute it and so in this way we're able to send skull code into python while making sure that all the references are being handled safely so another peak kind of under the hood this actually just ends up resulting in you have a common handler which unwraps your tracker object into a skull function and calls it and then for every type of scala function we want to convert into python all we do is create this kind of python a thin python wrapper function which extracts out all the args and calls your original skull function with it and so through this you can take any scholar function you want regardless of kind of what it's doing it might be interacting with other java libraries or anything and just send it into python safely another place similar to functions where we have a bit more of a complication is sequences so converting sequences uh from scala to python involves a various number of trade-offs so one kind of simple way to convert a scale sequence into python is copy all the data into a new python sequence so that's actually what skullpi used to do at first it would always just create a copy but this is quite expensive you waste a lot of memory because you're copying all that data over and that's not great for things like when you're handling large data sets um and it also kind of introduces this upfront cost where you have to convert all of the elements in that sequence into their python equivalence what's called pi is 0.4.0 which is kind of what we're discussing in this and that's the latest release of skull pie what that does is it also offers a python proxy conversion now what a proxy does is it's a very thin layer that sits inside python that references the original scholar collection and so every time a python library wants to access an element from that proxy it'll actually go all the way into the original skull collection and grab it on demand so this results in reduced memory usage since you just have a very thin layer sitting in python um and also eliminates the overhead of converting a bunch of elements for example if you only expect one of the sequence elements to actually be loaded so we can kind of see this in action in this example where if we have for example side effecting apply method in our sequence we see different behavior where the copy kind of loads everything ahead of time whereas the proxy when you print it out and python kind of iterates over every single element then you see those elements being requested and how this works under the hood is actually just powered by the function conversions we saw earlier so if we look at the difference between our skull conversion and our python sequence it's just two different methods that we have on each end for getting a specific item so you have apply in scala but underscore underscore get item in python and we want to kind of wire these to each other so we do is create a scala function which points to the apply method of that sequence that we want to convert then we convert it into a lambda using the existing logic we already have and finally we just wire the python sequence to use that apply method and so now every time someone wants to get an element from a python sequence it goes across the boundary into scala and grabs the element you want and then we perform a conversion on the scala element and kind of when we're returning that element back so that python gets a python value that it expects so under the hood it's actually quite simple and this is kind of the the result of skull pipe providing conversions for these very core primitives like functions is that we can just handle these types of proxies super easily just by defining a very thin layer lastly uh let's talk about maintaining type safety so type safety is kind of the core part about scala we really care about type safety and it's one of the great things that the language provides us so initially scholapi just provides some core type mappings between scala types and python types but in addition to this it also provides a complete type hierarchy that mirrors the types that exist on the python sets you have things like objects modules you have types for individual modules things like here's a none object in python here's an integer object and we have this entire type hierarchy that's available to you in your scholar code for interacting with python safely you can also define custom facades so for example if i want to interact with the random module which provides a random number generator in python i can define this using syntax very similar to scholar js a lot of skull pie is inspired by scholar scholarships because god just really figured out a great model for interacting with dynamically typed languages safely from scala so you define this at pi dot native trait you define what methods are available in this case we're just saying there's a uniform method that takes two doubles and returns a double we implemented with pi.native and now we can take our imported module cast it into this facade using dot as and now we can access all of these kind of typesafe methods and if we pass in the wrong types of arguments we'll get an error at compile time so now we can deal with python libraries safely what this actually results in under the hood is the synthesization of some type classes and that's how scalpi handles conversions between all of these without actually requiring any special kind of compiler tricks to handle these this is really powerful so when you're dealing with data science especially with regular python libraries you are often afraid that you're going to make a small mistake and run your big batch job and it'll just crash halfway through and that's not fun with scala pi you can write out of your data science operations and make sure and be confident that your code will actually execute as you expect there's also some really cool applications of these so scala pi is actually already uh compatible with scala 3.d so maxim who's a master student at epfl actually built this library called tensorflow.d on top of skull pi that offers shapes safe tensor operations so for example if we do a reshape it'll error out at compile time if the dimensions don't match up if the product of all the dimensions don't match the new set of dimensions that you want similarly it'll make sure that you're reducing uh on the right uh uh right dimensions of your tensor and so this makes it much easier to work with libraries like tensorflow while being safe about these types of complex bits and as a preview something that i'm really excited about that's coming soon to scalify it's not ready just yet but it'll be ready soon is the ability to auto generate type facades so python already has these awesome python type definitions uh in these pyi types that are being adopted by the community now so we can actually take advantage of these and uh use these python type definitions and generate scala type definitions so here's kind of a preview of what you can get where you even get the full type hierarchy where you say oh a python byte array extends a python mutable sequence of ins and it also extends byte strings and we get all the methods provided to us by python and you just run kind of a single command and scala pi will generate all these types for you so the hope is that this will make it even easier to get started with your favorite libraries um since you'll be able to use them in a type safe manner all the way from the beginning instead of having to move from dynamic to static over time so that's scala pi scala pi lets you use python libraries from scala code in both jvm and native code across these platforms so if you want to experiment in something like a notebook a jvm is great if you want something to be super fast use low memory low interop overhead skull native is great and your code will run identically on both platforms you get a seamless interrupt layer that handles all types of data structures conversions between them across both platforms and you get a fully types of interface that you can opt into over time to make sure that your production code is safe so you can get started today check out skelepi skalpy.dev we have a github and you can contribute there and we also have a getter community where we discuss what uh what's going on with all of our kind of applications of skull pie if you're curious about learning more about how skull pie actually works under the hood um i've actually been working on a paper which will appear at the skull symposium at splash this upcoming week which goes into a lot of the details of how we make uh handle all of these intricate details of mixing uh the different semantics of skull and python and i'd like to thank my advisor at berkeley professor koshiksen for uh guiding this project and getting it to be uh super precise with the way it handles all of these different semantics and now i'd like to answer any questions [Music]