scala.bythebay.io: Julie Pitt, Keynote: Exploring the Unknown With Scala
Recording: scala.bythebay.io: Julie Pitt, Keynote: Exploring the Unknown With Scala
I you I'm pleased to be here kicking off day two of Scala by the bay and I thank you all for being here today and I'm very thrilled that Alexei had me here I'm going to be a little bit of a misfit though because Alexei promised that I would be giving a talk about AI but I'm actually going to be talking about a primeval dilemma facing every single human being on earth and that is the trade-off between exploration and exploitation so I've been a software engineer for the past 15 years and I can tell you that I've come to rely increasingly on my past experience to guide my choices so choices about tools languages choices about design and architecture and just about every aspect of software development but there's a little bit of attention here and that is that we work in a fast-moving field the ecosystem changes underneath us year to year month to month and even week to week and so in order to stay relevant we need to explore but this takes time time that we usually don't have and when we explore new technologies we don't know at the beginning what kind of value we're going to get out of it and so when you compare that to fixing bugs where the value is well known you often make the choice to fix the bugs so today I'm going to be telling you a very personal story about my company where we face this very dilemma in our case we either needed to explore or go do something else so we're a scholar shop unsurprisingly and I should warn you that throughout this presentation Scala isn't really the main character it plays a supporting role and I hope that the takeaway message here is that although skull has gotten a lot of attention for being a scalable language building all kinds of wonderful applications with it you can also take it with you into unknown territory when I say unknown I don't mean something that no human has ever explored but something unknown to you so let me tell you the story it's going to come in a several chapter starting with our founding so we got started in 2014 and the founding team included myself Greg or Zell and Christian Kaiser so we got to know each other at Netflix and we worked in another startup together and over time we kind of decided hey let's pool our expertise together into a new venture why not do that and so we co-founded order of magnitude labs and we needed to first figure out what it was we were actually doing so we quickly decided on the frontier that we were going to be exploring and I'm calling this intelligent machines notice I'm not using the word AI here AI is kind of an easy term to use but unfortunately it means different things to different people so when I say intelligent machines actually mean machines that can learn how to perform a multitude of tasks that most life-forms find pretty easy but today's machine learning and reinforcement learning techniques find to be very hard so ironically one of these things is trading off exploration and exploitation it's really hard to get machine learning and reinforcement learning algorithm today to explore at the right time and so when we talk about exploration this is key to really building a model of the world that allows the agent to learn a variety of tasks you know tasks it might be as simple as foraging for food as simple as locomotion communication right all of these things so our hypothesis was and still is that there is a single algorithm that implements this learning and we wanted to go find it in fact we had read enough that we felt confident that the Nuggets were out there and that we could then apply engineering to it so here is our initial business hypothesis you know the emerging approaches are there we need to read the literature look at what's out there then we need to use our engineering background and apply that and then Here Come scale and now we've got some amazing problem solved that has real value in the market so we set to work building a prototype guess what we chose Scala so aside from all the usual reasons you might expect that skull is great we also thought that since we had experience seeing it at scale we knew we could take it from a prototype all the way up to a production application so we started working on our first prototype and I got very excited because I had used acha before and what we were building was kind of a neural net of sorts and what better model to create a a synchronous distributed neural net vinaka so you can imagine a single actor would be kind of like a unit or a neuron and the message is being passed we're kind of like the synaptic connections so I built the prototype and it didn't work so here was the problem I had already developed scalable applications in Scala so I could see what what you could do at scale and that kind of distracted me a bit I went and built the fancy version of the future before understanding the basics of what I was doing what is the domain that I'm working in and what is the problem space so the less of the first lesson was just because you can do something doesn't mean you should when you're starting out so then we went and built another prototype this was the embarrassingly simple version instead of a car with kind of weird bells and whistles it's this little bike it doesn't have petals it doesn't have brakes or any of those fancy things so this was embarrassingly simple and that all it was really was a set of case classes and some pure functions and these were all being stuck together I actually presented a scaled-down versions prototype at scala days in 2015 so if you're interested the links up there it'll be in the slides so we built this prototype and it kind of worked but it didn't really work and we kind of learned that there were some basic holes in the fundamentals of what we were building so we went back to the research and we couldn't answer these questions we couldn't fill these holes and we started to it started to dawn on us that the landscape looks more like this that we thought we could climb this mountain and build this thing and apply engineering to it but in reality if we're going to hold this hypothesis that yes there is one algorithm we're not on the right mountain range so we now needed to decide whether we were going to explore or go do something else so at a crossroads we all each had to do our own soul searching and Greg being a very reasonable person decided you know this is not what I signed up for so he went off to do other things and christian on the other hand was very enthusiastic about this problem space wanted to give it another shot and i did a little bit of thinking about it and I kind of realized something about myself which is that as a kid I was actually the one on the right I was really into art and music and all kinds of wonderful things but as I became an adult I was more like the kid on the left so I kind of knew that somewhere deep down there was this artist person that somehow got buried over the course of you know daily life and so aside from seeing that the potential value in solving this problem I also kind of had a personal goal which was I wanted to get back in touch with that artist and and kind of have this blend of engineering and art come together okay so little did I know that we would later stumble upon this amazing formula so don't be scared don't worry about understanding the formula but what I want to tell you about it is that this is essentially the value function that we believe all life forms must optimize in choosing behavior in choosing policies or a course of action and so the takeaway message is there's two terms in this formula one relates to exploitation and the other one relates to exploration and so we all have that creative Explorer inside of us and we need to get in touch with that person to do these kind of things so exploring is really hard and why do we not do it why is it so hard well the more experience you have in life the more sophisticated your model of the world becomes and the better able you are to predict a course of action that will lead you to valuable outcomes so if you know how to achieve your goal how to fix that bug how to build that service there's very little incentive for you two go investigate ten other ways that you might do that so how do we how do we push the teeter-totter in the other direction well we have to convince ourselves that the knowledge that we hold today the information that we have the model of the world that we have is inferior to what we will have in the future after we do that exploration and that's hard because we don't know how good our model will be we just know that there's some uncertainty that we need to resolve so I needed to kind of relearn how to explore something that I probably did really well as a kid and my first take at solving this problem was let's apply engineering best practices because if you have a problem there's nothing like engineering to solve that problem why wouldn't you have one single code base why wouldn't you have code reviews right why wouldn't you want to get rid of dead code have a project plan in a spec these are all wonderful things so I kind of started out with this mindset but in the end these things were not getting me the results I was expecting so I realized I was optimizing for something that was no longer relevant in my current context I was optimizing for getting code from a developer's laptop into production in a manner that was low friction low risk and you know minimizing technical debt doing continuous deployments right this is what I was optimizing for but at the end of the day in this world the stuff doesn't matter so what I should have optimized for was formulating hypotheses and testing them and rejecting the bad ones this is this is the cycle hypothesis testing where the friction needs to be removed and I was actually adding friction by trying to do these engineering type behaviors so I learned a few lessons after a lot longer than I care to admit about hypothesis testing so I want to just take you through each one of these in turn so the first one interestingly enough is you need to be able to contradict yourself what do I mean by that I can remember having technical discussions with Christian and he lets say it was on a Tuesday he would make a statement that was directly contradicting something we agreed must be true on Monday and I felt very disoriented by this I was as if he was changing the spec and it wasn't till I really thought about that I realized that I was acting as if what we decided on Monday was this back but in fact we were we're going on a mission to find the truth and in trying to find the truth you will be wrong and so you need to say things that are directly contradicting what you thought before so in other words when you encounter evidence that doesn't match your hypotheses go get new hypotheses right it's like imagine you're exploring a city you are unfamiliar with it you don't even have a GPS or phone or anything like that you're hungry what are you going to do you're going to form a hypothesis the hypothesis would be I think the food is over there so you drive that way then you run into a residential neighborhood contrary evidence what are you gonna do turn around go another direction so I there this is not as easy as you might think we had a problem where we needed to see what was going on in our experiments so I took it upon myself to build an application that we could use for all of our experiments to get some insight into and this this slide here is actually the talk that Alexei mentioned my 2015 talk on Scala GS so this application you know hooked into the experiments collected some data and because I was trying to be low-tech spits it out into JSON I have this acha HTTP server going on with a scala jas front end and then d3 visualizations right so I have this kind of whole pipeline mapped out so there's nothing wrong with this app except that it adds friction to the hypothesis testing loop and the reason is that our hypotheses were not as stable as I had imagined these experiments were changing more than I anticipated when I built the app and so different kinds of data came about and that required a lot of work to integrate a new experiment and furthermore new visualizations were needed that I didn't anticipate so that was more work so there is friction added and what this meant was I was afraid of saying I was wrong and test in rejecting the hypothesis I was testing and trying a new one so we had to come up with a compromise how can we get insight into our app but then still not add this friction so we came up with this very low tech we had some very simple functions that printed out you know SVG files and stuck it in an HTML page and we could load that up there's no application needed and this kind of freed us up from this idea that oh we had invested too much in the current hypotheses so lesson here is the more that you've built the harder it is to admit you're wrong so be careful about estimating your uncertainty about your hypotheses and gauge your estimate gauge your investment accordingly okay so another way to look at it what are signs that you're investing too much when you have to keep adding complexity to prove that you're right then at that point you should start applying Occam's razor how many of you guys know what Occam's razor is okay good most of you just for if you're not familiar with it Occam's razor is really just the statement that if you're dealing with multiple competing hypotheses you want to select the hypothesis with the fewest assumptions so at this point when you're adding a bunch of complex so you want to step back and say what are some other alternative hypotheses that we can use to simplify what we're working with okay so we kind of got over this message of okay hypotheses are unstable I'm going to be wrong this is not a speck okay but now we needed an environment to work in we ought we like to think about our development environment or build process our continuous integration and all these wonderful things but you know guess what this is not what we're doing here we need to run experiments and in order to run experiments you need a laboratory so how do we set up our laboratory first of all tools are essential you need to have the right tools and you need to have enough tools because if the only tool you have is a hammer everything is going to look like a nail so I have a corollary to that which is that lets say you add a screwdriver to your toolbox now all of a sudden you can tell the difference between a screw and a nail so I kind of came to realize how important tools are to this whole process you might already be thinking yeah of course you form hypotheses you do experiments you form hypotheses you do experiments but you start to learn patterns as you go and realize that you need some tools and that when you do an experiment you might add a tool here that then helps you do the next experiment and furthermore when you add new tools you might start seeing the world in a different light so you might actually realize new hypotheses I mean there are several times when we kind of added a tool to our toolbox and in our case this is mathematical formulas and techniques and we realized at that moment hey you know that problem we had a month ago and we couldn't answer that adding this tool actually helped us see the answer so how do we think about the toolbox I think a good way to think about it in this context is that it's really an index for what you know it's a reminder of the techniques that you're already familiar with so that when you are going out to solve a new problem you don't need to go back into first principles and say hey what do I know so when it comes to what should you add dear toolbox guess what this is code code is cheap so be greedy and when you see something that might be applicable at some point go ahead and add it to the toolbox this took some time for me to get over because I was used to thinking well dead Co you know it's not being used this is going to add you know friction to the process and people are going to be confused not so in this case you also want to make sure that when you're testing hypotheses you can be lazy that the tools are within reach there's they make very few assumptions and they're versatile and then lastly don't over engineer them you know going back to the the insite app example for where we were at that was over engineer so I made my peace with print line it's okay so what did we put in our toolbox we actually since we're dealing with math we have a bunch of primitive operations on floating point values just collections right pretty much untyped if you guys have ever used python and numpy it's kind of sort of analogous to a mini version of numpy right it's a bunch of functions you can use but you don't really know what should be used with what so that's how we started but eventually we kind of started to be able to name things and at the time we can name things hallelujah we're using Scala so now we can start adding types and we can start adding methods to those types and build on top of that so now the tools come with an instruction booklet we have types so the tools the types really make sure we use these tools correctly so I would say this is kind of a bonus on top of some of the popular scientific languages and frameworks right like MATLAB and Python now we have types hallelujah okay so we have tools but we need a work space to operate in the tools need to be really accessible we need to not have a lot of heavyweight process to get going on an experiment and we need to not have a lot of distractions and keeping the environment going so we were very lucky in the timing because at this point in time ammonite comes out how many of you guys know what ammonite is a lot of you good how many of you guys have used it before fewer how many of you use it on a daily basis fewer okay so i won't talk a lot about all the wonderful things that ammonite does and all of its features Howie there to call it to call you out has some great talks about it and I think he's going to be speaking about it right after this but for us kind of the big win was we can easily import our toolbox into ammonite and we can we basically use it as a rebel so there's there's several different tools within ammonite there's a rebel there's also Scala scripts so if we wanted to go out and test something like real quick hey does this function behave like I thought it would or hey what happens if I do this you know import your tool right a light or two of coat and bam you have the answer there's no setup involved if we wanted to do something more involved let's say repeatable we can write a scala script it's just a plain old text file and run it using ammonite as a script 0 setup involved so the argument in my mind that Scala is had to heavyweight or has too much overhead to get going I think that's actually fading away and that you can do these quick and dirty things using Scala so you know we still have the because we're in Scala we now have the option of adding more process and more weight to it when that makes sense we can still create an SP he build we can still do continuous and integration and all this wonderful stuff okay so now we talked about the basics of exploration but how do you do it as a group what if you have more than one person and the thing that I needed to figure out was at what level should we be in sync and my first instinct was well we should have one code base and we should do code reviews and of course you know why wouldn't you but what I kind of came to land on is really you want the ideas to be merged and in sync but the code can be forked it doesn't mean the code is just completely orthogonal but it means that it doesn't have to not every single person on the team has to be on the exact same code so imagine this right think back 150 years before we knew how to build flying machines well there were many different form factors proposed for flying machines and they were all wrong there's a there was a lot of ambiguity going on so there was a number of hypotheses that all seemed equally plausible imagine if someone had said hey guys this is the form factor we're going to test it gets you guys can make your tweaks but this is where we're going to go against we never would have created flying machines so if you require a single code base too quickly you might actually downplay the level of ambiguity in your environment you're sending a message to the team this is how it is when what you really want is to keep your options open and allow ideas to flourish the other downside to having a single code base when there's this much ambiguity is the discussions naturally slant towards how how to do something how should the code be structured what are the coding styles and this really takes away from the high level of what talking about how feels really good because we get a sense that hey we know what we're doing we're just kind of going to figure out how to do but you know if you're making yourself feel good but you're not actually making any progress that ends up in the long run hurting you so what should you share well you should share you know what do we think the principles of flight might be what are the hypotheses you want to share these as a group often and eagerly because as a group you can perform very powerful thought experiments that might throw out the worst ones and the ones that survive this group discussion are then the ones that the group is going to go out and test you also want to share tools right if you think of tools as this index of what you know well guess what you adding a tool to the toolbox now means that the team as a whole has their knowledge enhanced so you might not think a tool is relevant right away but your teammate might think oh my gosh this is going to solve my problem and lastly of course as they start to emerge you want to really be in sync on the requirements the whole process of doing this explorations getting to the point where you think you could write down the requirements so that's going to happen slowly over time so contradict yourself you're going to be wrong you need to optimize for hypothesis testing in a laboratory environment not in a production environment and let the ideas come together but don't require the implementation to come together so just to briefly talk about this I mean at the end of the day we want to build a real practical application of value we can't get there by exploring you can't do that in forever for forever you need to execute and so kind of we've done several oscillations i would say between exploration mode or creative mode and execution mode so i call these modes because each one of these requires a very different mindset and you can't really keep that same mindset both of these at the same time so we kind of have this loop where we're iterating between sort of taking a step back being creative and executing so we need to have a good algorithm for mode switching and it's quite simple I think but hard first you need to know which mode you're in and then you need to look for signs that it's time to switch and lastly you need to give yourself time because it takes time so creative mode is what we've been talking about all along which mountain should i climb in this mode the hypotheses are incredibly volatile and you're really kind of looking from the 30,000 foot view execution mode is more like I'm just going to I need to climb this mountain and really for what I've observed is the really only difference between creative and execution mode is how stable the hypotheses are in order to execute you must have stable hypotheses because you're looking at the goal and you're trying to get there so in terms of when should you switch if you're in execution mode you need to you need to know when to become creative again and this goes back to contradicting yourself you still need to be able to contradict yourself we just hope that when you're executing this happens a lot less frequently so when your assumptions are wrong and you start to encounter this evidence that's contrary then it's time to change your your mode and so you know technical debt and mounting complexity could be a sign that it might be time to go do this and given the environment we work in lots of things moving pretty quickly we should consider doing maybe a mini creative mode pretty often and then execution mode is a little easier because you kind of naturally want to do this when you can describe the goal you you can you think you can get there and you can't find any counter evidence that's when you need to go execute and then the last thing is give yourself time so this one I don't have an algorithm for more of a heuristic and what I've observed is the farther you have to zoom in or out the longer it's going to to switch so if you're going from optimizing bytecode to let's say coming up with some kind of five-year plan you might need days to switch or even weeks if you're fixing a bug and you realize need to do refactor maybe in an hour you can switch but do recognize that it does take some time so just kind of briefly summarize what we talked about prototyping build the embarrassingly simple version first understand what it is you're building don't be tempted by all the wonderful things you know you can do in Scala before you kind of understand what you're doing make sure you're optimizing for hypothesis testing this is not optimizing an assembly line of code if you will and then lastly make sure your mode switching so just kind of taking a step back I'm going to bet that a good part of the reason why you all are here is because you want to explore you want to explore new technologies new developments in the ecosystem and you're probably going to come away excited about all the things that you learned but what I want to drive home here is that that's step one this is the beginning of exploring these new things that you're learning about we can I hope that I've convinced you that Scala is a great language for just trying things out you don't need to go into a heavyweight process of setting up a build and setting up a project in order to tinker around so I encourage you to take some time with what you've learned and what you're going to learn in this conference and start playing with it see in exploring your curiosity so thank you very much for your attention and if there's any time I don't know then we'll take questions [Applause] ok so back right here sorry there's well known sorry sorry upper confidence bound in terms of you know so that that would be an interesting so in terms of like actual metrics yeah i mean i didn't i didn't bring that in i think you know that that's another interesting one to bring in right we've been kind of doing this kind of from a feel perspective from a seat-of-the-pants perspective but there might be interesting ways to bring in sort of a more regimented approach to kind of knowing hey when do you have that tight confidence bound and when should you explore so thank you for bringing that up so um I think that's correctly I think you were saying that everybody in this room should skype either exactly this was a pitch for you miles so everybody go fork the Scala compiler mauritius thing or modulus all right [Applause] you