Scale By The Bay 2019: Jeremy Smith & Jonathan Indig, Solving the Scala Notebook Experience
[Music] everybody thanks so much for coming to our talk I'm Jeremy and I'm Jonathan and we work on machine learning tools and libraries for recommendation systems at Netflix we're gonna talk to you today about poly notes so poly notes is a polyglot notebook environment which we built from scratch and how many people here have used a notebook tool of some kind okay so a lot of people good for anybody who hasn't used a notebook tool for a notebooks basically a document containing code and the results of executing that code and typically some text describing what the code is doing and it's useful for data science exploration and experimentation of among other things so notable examples you may have heard of or used are Jupiter or Zeppelin or Mathematica so given that those tools exist already why did we build poly note and the reason is we we noticed our scientists were or some of our scientists were avoiding Scala notebooks for their experiment a ssin needs and instead they would either prefer python-based notebooks or they would just abandon notebooks altogether which is a shame because it has a lot to offer in terms of productivity and iterating on on experiments and it's a problem when they choose Python because our infrastructure is largely based on Java and Scala and it's much easier to productize a notebook when it's already written in the same language it's using our platform in our libraries already and if you port from Python to Scala you can that's a good opportunity to introduce bugs or other subtle differences that may affect the outcome of an experiment and the reason they were doing that is just a pain to use Scala and spark in a notebook and some of the specific pain points were just the editing experience in general I think so Interactive autocomplete is practically a necessity when you're editing Scala and that's just because it helps really helps you find api's imports and discover type signatures of things and so that's really mist in a notebook tool which typically doesn't have that that feature it's difficult to find compiler errors because scholars compiled language people are used to having in their editor a little red squiggly when something doesn't compile in a lot of notebook tools don't give you something like that the dependencies in a scala project are a much greater number and ver and more varied than typically in a python environment so a lot of our researchers for example might may have dozens of transitive dependencies that they need for their experiment and those dependencies can often clash with sparks own dependencies which leads to you know researchers spending a lot of time building shaded jars and that's time that they could be spending innovating on the algorithms instead so just some context here to show you that we really tried not to reinvent the wheel with this so the original idea that we had was just an IntelliJ plugin that would run notebooks for Scala and IntelliJ while leveraging IntelliJ zhh built-in editing capabilities and we made various attempts at that over the years and and they didn't really pan out that well for us and we'll talk about why in a little bit and I found myself grumbling about these issues over time and at some point I thought you know if you think you have all the answers why don't you try to implement some of this stuff and and see maybe it's harder than you think so so I started this proof-of-concept as a side project sometime in August of last year and I started Sheryl abaya tober I had a working proof of concept and I started sharing it with some researchers and I got some positive feedback and they said show it to other people so I did that and pretty soon I actually had a couple people using this tool and around the same time jonathan was also working on the Scala notebook problem from a different angle so he was working more on on evaluating if we could make an impact on the Jupiter side and I heard that he was doing that so I showed Polly note to him it was immediately clear to me having tried these other approaches that Polly note was the way to go so so by the start of 2019 we actually had people using this for real work and at that point faisal shoutout to faisal took a gamble on it and started deciding netflix would start investing in this this product to see where we could take it and and Jonathan came aboard to work on it and really started to take shape after that and then ten months later we announced it as open-source so we wanted to talk about some of the things we learned first in this process and then Jonathan's going to show you some demos of it so so first of all I mentioned that we tried to make this an IDE plugin and it didn't really pan out so what went wrong why couldn't we do this in like as like an intelligent or an eclipse plugin or something like that and and the reason really is that a notebook has to display a lot of rich sort of heterogeneous document like content you know things like rich text graphics plots law tech equations data tables and even some dynamic scripted interactive content and the only standardized really pre-baked mechanism for that is HTML and that means you can either support HTML in your in your notebooks or restrict the richness of what it can display and that leads to you can really either embed HTML into your UI or you can embed your UI into HTML and it for us to turn out to be easier to do the latter because there's not really a feature complete HTML implementation on the on the JVM at least not one that can compete with what what Firefox and Chrome can do and interestingly a lot of other IDs are also now getting embedded into into HTML so maybe that's a trend there but those tools aren't aren't widely used on our team so we decided to try this out with a standalone web app and interestingly jetbrains now has their own IntelliJ based product for doing notebooks which I've heard about so somehow they've succeeded there where we failed a couple of times but I haven't I haven't seen it yet but it'll be interesting to see how they've done that so definitely check that out and why not do a Jupiter kernel so this was the other other approach that we explored because after all Jupiter is heavily used and invested in there's lots of people at Netflix that that contribute to Jupiter and and a lot of people at Netflix that use Jupiter and were you know we continue to invest in that ecosystem with things like interact in paper mill and and we would never be our goal to displace that so why not just work on a Jupiter kernel instead to integrate with our with our other investments there and we did we considered may be contributing to almond or trying to adopt that and see if we could use that to solve some of our problems and that almond actually since then has come a long way and it's really stable and it works really well no thanks to us unfortunately but if Jupiter works for you should definitely check out almond but while we were investigating that we came to the conclusion that the kernel was really only part of the problem for our use case so a lot of the features that we wanted to implement and that were important to us it's possible they could have been implemented as Jupiter extension but Jupiter's model isn't really equipped to efficiently support some of these features that we wanted and the reason is the document stayed in Jupiter is all client-side so any edits you've made to your notebook exist only in your browser until you hit save and that's that's limiting and causes some various issues with the experience for example if you want to autocomplete you have to send the entire content of the cell to the to the server and that's not something you'd probably want to do on every keystroke because those you know extra couple hundred milliseconds of latency get really noticeable when you're doing interactive autocomplete and there's other issues with the model like what happens if you start running a 10 hour notebook and then you close your laptop and go home it's not super straightforward how to deal with that in Jupiter and like I said some of these things can be handled with extensions but we really wanted to do was to build it with a model that the document lives on the server and only edits are sent from the client to the server and when it turns out when you use a model like that you get some other cool things like collaborative editing more or less for free so I'm gonna hand it over to Jonathan now and he's going to talk about what makes Polly Note different from other solutions thanks Jamie I'll attempt to show some videos on here so cross your fingers for me over there so I'll be discussing some of the features of Polly no to really showcase what we have to offer and we'll dive right into the editing improvement so Jeremy already kind of talked about these quality of life ID like autocomplete parameter hands and squiggly lines very important and here's a little video here pre-recorded that should start yeah so I'll start my past self will start typing some code and I can barely I'll see it okay and so we're just gonna mess about with a concurrent hash map and the particular example is not really important but we just show the autocomplete parameter hinting and as as I type here I can talk about this little code editor which is based on Monaco which is open sourced by Microsoft and it's the engine and it runs running behind vias code as well so it's awesome that they open-source this so that we can just drop it right into our app and so let's run that so okay great so we've got our little value there and the next thing I'll show is just will generate a syntax error and show us a little squiggly line okay there we go and so we show both okay you can see it both the squiggly line there that's out of my view and also the air and we see that on the air there's also line five there so you can imagine in a very big cell how this could come in handy often times we see our researchers with a single cell that can span the entire webpage and hunting for errors and that is really problematic if you don't have the nice little squiggle and now I'll show this rich text editor that we have up there has some improvements as well like just the simple tool bar where you can bold and underline and hope everyone can see that and also you can type in law tech but I didn't do that here so hopefully you guys got a little glimpse of what it's like the code with a poly note and next I'll talk about how we tackle the notebook reproducibility problem so earlier when Jeremy asked a few of you guys a few of you raised your hands how many of you have tried to use a colleagues notebook only to find that it didn't run on your okay that happens all the time for us as well basically the cells got executed out of order something got deleted or modified and then the state you see when looking at the notebook on their machine doesn't doesn't work out if you just try to run it from the top and the reason for this is essentially that there's a disconnect in the notebook model nobody give you this UI where you see a whole bunch of states at once of all these executions that happened at various times you get to see them all at the same time but the but the repple that's actually interpreting the code only sees the code one at one at a time so just every time you press execute we send everything to the repple it executes it doesn't know anything about where the cell what salad is what's going on in the notebook and all that and then the problem there is we've got all this hidden state that we need to keep track of and the way Poli note solves this is simply by not being replaced and so we'll talk about what the implications of that are so here's a simple example very trivial notebook we've got foo we're sending it to one we're gonna print it out and then we're gonna set it to two and I yeah okay let's try and see what happens so we're just going to run through this okay so sadly saw we ran saw one printed Fuu as we expect it printed out one then we redefined Fuu to be to anyone who's used Jupiter before what are we gonna print out here okay let's see what happens all right it was one again not two but maybe like it didn't take let's see maybe we just the execution didn't work or something let's find out it's still one so what's going on here essentially in Paulino the cells ordering inside the notebook actually has a semantic meaning and essentially a cell can only receive inputs from the cells above it and anything that happens in the cells below it can't affect it so in this case this cell to here can only see what happened in cell one cell one of course has foo equal to one so we don't really care about what happens in cell 3 and so on now doesn't mean that this is immutable of course you can always edit cell one and then see something else or we can insert a cell above okay and as you would expect we redefined foo because it's above the definition kind of takes and then we print out so I'll take questions out there and sorry about that okay so that's about all we have to show and then we do keep track of the state as things happen so if we delete this new cell we made and then print again we're back to one so hopefully you can see that these changes help improve the reproducibility of notebooks because essentially as the users iterate on their poly note notebooks it's always making sure that they can run again from the top so now we'll talk about visibility visibility by which we're referring to basically giving the user a better understanding of what's going on behind the scenes of the notebook so we have a few kind of UX features that let you kind of feel like you have more control over what's going on more information that can help you as you're editing so this is a bunch of code right now the actual contents aren't that important but I'll run it through and we'll see what kind of information point-o gives when you execute and you might have noticed this in the previous demos okay now I was supposed to go back to the beginning here yeah I'll just let it play through and then okay so now we're gonna run through again and I'll pause and do some explanations okay so the first thing to notice is we press the execute all button which of course many other notebook tools have and we immediately see that cell one is running and cells two through four there are queued over on the on the right hand side and we see the order in which they're queued which is something that you don't see in some other tools and we also can there's a little highlight here which shows exactly what line is being executed and it's also mirrored over here and when I unpause you'll be able to look at this highlight here and see as it moves through each expression as it gets interpreted executed okay so now we've got two a little spark job that doesn't do anything of interest but what we can see here is this little progress bar appeared underneath cell 1 which shows that we're and you know the job ID what's going on and then what stage is running as well and now I'll keep going for a little bit and we'll see how as the cells get executed in turn their progress bar disappears okay now the the next thing to show here is this symbol table right now because no cell is selected we only see the sparks session which is what Paulino provides for free when you start up a spark notebook but as we start selecting cells we'll be able to see the state the symbol table change reflecting the state that's available to the cell that's being selected right now okay so we selected cell one here and we see that the symbol table has changed now there's a dark line here and we can see these two variables that are defined inside the cell above the light and of course that old spark variable below and what this line delenn is the outputs above from the sorry yes the outputs above from the inputs below and as we click on for example cell 2 we'll see that now the output is why because that's the fine but all the state available to it the inputs are below okay of course so hopefully these UX equivalents go a long way to help you understand what's going on behind the scenes now of course it's in the name poly note so let's talk about polyglot every cell in notebook can be written in a different language and variables and state can be shared between them currently we support Scala Python and sequel cells with sequel essentially being spark sequel so I'll focus today on just scholar and Python and so here's a little example of a polyglot notebook where we have this first cell here is written in Scala and the second cell is a Python cell and this is just doing a very simple thing where we do a bunch of you know some data generation uh preparation in Scala and then do some plotting in Python and this is one of the main use cases that we we find value in this polyglot support because people just like matplotlib but people also want to use our platform which is written in scala to generate data and aside from map la lib another thing people use python for a lot is of course machine learning like tensorflow and things like that so this lets us lets them use the same libraries the Scala libraries that we already have for data prep but then you know bring tensorflow psych you learn whatever they want so we'll just run these two cells now and we'll highlight two array there we'll get back to that later and then ran we have a nice plot and then okay we have a mouse Topher data to highlight that so that data variable here is the one defined up here from Scala and the reason I bring up the two array conversion here is just to show that this Python and Scala in trop it's not like super magic or anything you have to do a little bit of playing around and it's under the hood we use a great library called JEP which stands for Java embedded which runs an embedded C Python and Java through the day and j'ni and that's doing the heavy lifting of translating variables back and forth and the reason it's better to use an array is because you know arrays are these data structures that everybody knows about whereas if you had to send a send like a Scala seek or a list over to Python it's not really gonna work in a nice way so here because it's an array in the JVM world it just nicely slots into a list here in the Python world and we think this is one of poly notes most interesting and powerful features and now I'll talk a little bit about the dependency in configuration management essentially this you know nothing super amazing here but just by saving the dependency info and configuration inside the notebook itself rather than being kind of a function of the environment as it often is we're able to get a lot of leverage and just be able to operate these notebooks that lots of users are using and make them more portable so they can be shared and so I'll just show a little bit of the UI so we have this little UI here that allows you to put in packages like maven coordinates or even as three jar locations things like that if you do use coordinates we'll use course here to download those dependencies and also they're transitive dependencies for you and we can you can define IV and maven repositories here and of course we can exclude guava from our programs using this here and this also works for the Python as well so this you can make a pip dependency here like matplotlib say and then here you can put your company's Pitre pi PI repo as well and finally we have a area here for the spark configuration and the nice part about aside from having on making this portable is when you put the spark configuration in the notebook itself we can read it before we launch the notebook kernel which lets you set the driver memory which has been a problem for us at least when using other notebook solutions so we don't have to ask users to edit shell scripts in order to change that and so finally having all this in the notebook file which is essentially just adjacent file allows us to interoperate with some of our other internal tools at netflix so one of the things that we can do is we can have a tool that just generates a notebook that a user can download just hop into and we preset like an environment let's say a jar where something went wrong in production or something like that who's already present and available so that really helps us out as well and finally I'll show some are built in data visualization so we have this data inspector here that I'll talk about and what we can do is so this this cell here we're just generating some some pretty looking data we're gonna plot it down there and then in the bottom which we're actually going to go to first in this video will show some interesting a collection with some kind of structured data so we've got a bunch of pets they're all dogs and we can see just what the schema looks like immediately after we run this and we can and and this is kind of you can imagine is a pretty useful when you've got like a very complicated nested data structure that you have to deal with up there now let's talk about this graph so again it's just this little this function that generates a bunch of data and the key thing to note on this is just that we also add some noise which I'll show later and this schema isn't particularly interesting but we have all these little quick access buttons here that take you to both a tabular and a plot view so we'll click on the plot now and then so we can have a plot but let's first take a look at the data so we know how to plot it we can just page through it which will stream that some data a page through the data basically okay it's a bunch of numbers that doesn't tell me much let's plot it will do a line plot just like the one over there and we can set little quartile view on the Y's and then just drag over the x-axis and we'll press the will start plotting and this takes a little bit a little bit of time just because we're actually running the spark job right now behind-the-scenes to stream the data over to the client so it's a little hint of that over there boom we've got our data looks just like the one we just saw let's save it to the graph sorry - the notebook and so now we've got this graph sitting in our notebook and this graph if you notice here it's actually a Vegas Peck type cell and what that actually means is we can show the Vegas Peck and see that we just have the Jason right here with a little bit of our little secret sauce to actually pull the data out and aggregate it but what this means is if you if you have some Vegas Jason that you want to plot you can just paste that in there and then it'll plot for you now that's been a whirlwind tour of plenty notes features over to Jeremy to talk about what we learned building it Thanks so I work mainly on machine learning libraries so for me it had been a while since I've done any sort of full stack web app development and given that people including myself were kind of surprised at how quickly the POC came together and then it went from a POC to an actual useful tool that dozens of people voluntarily use for their day-to-day work in like less than ten months of to people's part-time attention and not to brag about it but I'm pretty proud of the functionality that it has given the small investment of man-hours we've put into it so far so I thought we'd talk a little bit about the process of building it and some things that we learned mistakes and successes in case that's generally useful to anybody so the first thing is obviously standing on the shoulders of giants and this is not new at all but I just wanted to call out some of the amazing open source software that that Polly Notes built on the back of there's Monaco like like Jonathan mentioned which is the the same code editor used in vs code and Microsoft's actually open source that as a JavaScript component you could just drop into a web app HTTP for s and FS - these are two amazing libraries that are built on cats and cats effect and I was just blown away by how easy it was to just put together a server and a couple lines of code it's got WebSockets support out of the box and a really reasonable model for working with the WebSockets and it just works you know it which contrasts with some of the heavily configuration oriented JVM based server frameworks that that I was familiar with and we actually didn't have a config file at all until a couple months after that so that really blew me away Zeo we adopted a little bit later on and we'll talk a little bit more about that but that brought some really amazing innovation to how we could structure our code so kudos to Zeo team as well ska deck let us just build an ADT and then BAM for free we have a compact binary protocol for those messages and that's really one of the reasons I think that probably don't feel so fast course here like Jonathan mentioned for for dealing with those dependencies anybody who's used Ivy would know how much of a nightmare that would be without course here so shout to them Circe we're using for configuration files and to actually read and write the ipython notebooks which is the the notebook format that we're using and and Jonathan mentioned Jeff without which we probably could not have added Python support so easily so a big shout out to all those projects and you know I was just I was surprised almost and pleased with how rich and and mature that the open source ecosystem in Scala is now it's really come a long way I think in the last few years and I feel like these libraries almost gave us superpowers you know it was shocking and I and I say shocking not to brag but because I was actually kind of shocked myself about how quickly we were able to build new features and fix bugs and our users were impressed by that too I think that's one of the reasons that we sort of developed some diehard supporters of our tool and and I was really pleased with how welcoming the communities around these projects were as well so we would always be jumping on getter with stupid questions and there always be somebody some courteous community member there to help us out so you know it just a bit a big shout out to all the people that work on scala open source or contribute to it it's it's making an impact so so you should know that the next thing is that IO with functional programming is really awesome so I had used it in some libraries I was no stranger to functional programming but I hadn't had the opportunity to build like build an application from the ground up with with something like cats effect and I was a little bit skeptical about the trade-offs of doing that but doing it I was just blown away by the advantage that it provided we had to cut some more slides more material about that so feel free to talk to us about it in the hallway with zio we adopt it or I guess at CEO we adopted it about seven months in and the reason was mainly to get interrupted by Oh for free we have those tasks that are running there's actually a literally a button to cancel a task and when somebody who does that we want to do the best we can to to cancel it even if that means interrupting a thread and so zio gives you that out of the box and we could get rid of the the hacky stuff we had in place to do that with cats effect but if we came for an erector bio we stayed for the environment mechanism I'm sure that'll be covered by other people in more depth we had to cut some material about it so again feel free to ask us about it so uh the last thing I want to talk about real quick is just what we learn from open sourcing this because this was the first project that I had open source under sort of a well known name being that you know the Netflix open source an umbrella so it's actually been open source since the beginning as a side project but in October we made a blog post and sort of exited stealth mode on that and it was a pretty small project from just like me and Jonathan are the people working on it and we didn't expect to get much attention so we felt kind of okay about maybe taking a little victory lap in our blog post and patting ourselves on the back a little bit but we didn't consider how that blog post would look to people without the context that we had like we know the scope of the project and we know it's like no big deal but you know we didn't think about the fact that this gonna be a blog post saying you know Netflix releases this thing and it's gonna give people the mental image of like Reed Hastings storming into the boardroom and saying oh we have to release Paulino to open source and obviously nothing like that happened but we didn't really think about you know how that would appear and and in retrospect we should have been a little more bit more transparent in our blog post and it got a lot more attention than we expected and it was mostly positive you know that the only negative stuff were people complaining about the logo but it it did get reblogged a lot with a bunch of sort of misleading headlines so just for entertainment purposes we're going to just go through a couple of those so this one's fun what you need to know about Netflix is Jupiter killer so Poli note is emphatically not a Jupiter killer it's not intended to be a Jupiter killer and Netflix is always going to be big into Jupiter I promise you that so you know if this headline was a little bit bothersome actually because a lot of our colleagues contribute to Jupiter and are active in that community and yeah so I could imagine some some concerned emails here's another one to just more Jupiter confusion again we're not really related to Jupiter but this next one is my favorite and then I promise I'll stop did Netflix just killed data bricks right so obviously the one-word answer is no and I think the article also probably came to that conclusion but you know this kind of clickbait can can really paint our intent in the wrong way so you know I guess the lesson is you know hopefully our friends at data bricks got a good laugh out of this if they saw it but the lesson is kind of you know be more considerate about you know how your your posts will be conceived or perceived by the people who don't have that context so this would have been the real headline if there was one but I hope you know we gave you an idea of what poly note actually is and come check it out poly note slash poly note on github PRS are welcome and again we had to cut out a bunch of stuff so feel free to talk to us about that in the hallway I don't think we have time for questions but yeah [Applause] [Music]