Scale By The Bay 2019: David Andrzejewski, Reliable Machine Learning

Scale By The Bay 2019: David Andrzejewski, Reliable Machine Learning

Recording: Scale By The Bay 2019: David Andrzejewski, Reliable Machine Learning

[Music] all right thank you all for coming thanks for that we're gonna history's for having me my name is David Andrew jetski we're ketsu mo logic and I'm here today to talk a little bit about reliable machine learning and kind of broadly what are some of the problems and what are some of the solutions out there first really quick just little bit of context just if you're familiar with the video advertising this notion of a pre-roll this is kind of what you're seeing through right now just some context what my employer what we work on is a cloud-based data analytics platform collecting logs metrics on all these kind of events generated by your software at a large scale and doing some fun kind of data analytics with that the basic idea is you can kind of capture all of this information use it for ops uptime security things like that so if you are running a software oriented business as I imagine many of you are this is something you might want to consider from the engineering side there's quite a bit of interesting data stuff going on there happy to talk about it sometime grab me in the hallway so probably the focus of this talk ultimately I don't think it's super controversial in this audience to say software kind of eating the world software all over your software and everything software your phone however it's also probably not controversial to say that very often the software is not so great there's all sorts of bugs vulnerabilities dependency problems supply chain attacks different kinds of failures if anybody tried to use the link anybody local that PGE sent out in the event of the wildfires recently and that didn't work because their website had fallen over if you recall the healthcare.gov launch you know clearly you know we as a civilization I have not quite 100% nailed the efficient creation and authoring of super reliable software and now we are also going to add machine learning to it and so now we kind of have this new sort of tool in the box new ish and it's kind of getting democratized and pushed out into more and more different kinds of software and how is that going to make things easier or not easier and I would claim for the most part at this stage it is not really going to be easier so what I'm going to try to walk through is just kind of a survey a lot of the different kind of pitfalls and some of the tools that are out there from an open source or commercial or services perspective that are at your disposal to potentially grab to kind of help deal with these new challenges so our ml system is challenging right and I would say I'm not going to dig super deep into this but this talk is going to be full of a ton of references so when I eventually we share out the slides the beep most of these things are linkable searchable kind of stuff but if you talk to many many many folks these are kind of considered sort of iconic research papers out of google in the cataloguing of the sort of pitfalls and problems that you can kind of hit trying to build practical data-driven machine learning driven systems right you are going to have new ways of coupling your system you're going to have new feedback loops potentially when you're experimenting you're going to have a lot more iteration dead code that prototypes changing the features there's a whole kind of avenue of new ways to a couple of things and so that's kind of the high interest credit card of technical debt paper which is a well worth your time and kind of a sort of spiritual sequel to this is this like ml test score paper and a lot of these ideas are going to come up throughout this the survey of what does it mean to actually have a tested ml system and this kind of extends all the way from your data the features the future generation code your actual modeling code your modeling serving deploying code testing the tests everything like this so as kind of a starting point this is a great place to start to really understand okay you know what what are some of the challenges and we'll dig into it a little bit more and yet another one kind of on some parallel track is you know rules of machine learning these other kind of best practices and if you read this what's really jump out at you there is very much emphasized really just stitching together and end-to-end thing simplest possible way first and then kind of building up on top of that and that's going to kind of come again as sort of a consistent theme in this discussion as well is if you cannot sort of make the very very simple thing actually work or actually be kind of reliable or test that when you create the most sophisticated possible thing with the really elaborate feature logic pulling from five different data stores and feeding into a you know brand-new deep learning model you're going to have big problems and so there's kind of a sort of agile almost kind of approach advocated in this paper so let's let's assume that we're going to buy that it's challenging and now while it's like try to understand a little bit how we could think about this sort of given this given this conference in this venue you know one way to think about it is kind of from a functional perspective and another talk and slide sides I'd really recommend talks about kind of how types and tests compare in terms of constraining or defining what constitutes a valid implementation of a function and this gives you like a very quantitative way even to think about what does a given function do it again at this audience probably that the type sort of approach is going to be kind of compelling but if you take this first function that goes from a string to a string how many different possible implementations could you have ludicrous number and you're going to rely probably on unit tests and other sorts of constraints on your behavior to try to narrow it down to some subset that actually does what you want whereas by let's say that you have some kind of you know like structure and you have a second function that just says is it weekend and it just takes something from the enum structure and maps it to a boolean this is obviously super trivial example but it's meant to kind of sketch out the two extremes of this continuum by kind of you know looking at it from this sort of type how many different possible implementation perspective you can say that being confident in the correctness of the second one is going to be a lot more straightforward I mean confidence in correctness of the first one so let's put on our you know types first test hat and look at machine learning and so one kind of classic way to think about the standard supervised setting of machine learning is that you have a function a training function that map's a collection of training examples so this could be input X as your email output label Y is whether or not it is a spam and you want to learn a function that maps from new X that you might see new emails to predicted label of spam or not spam for those new emails and let's we can kind of take the same tool for thinking about it from the previous slide of how many possible implementations of that function are there and actually an entire sort of swath that I if you're interested in this kind of thing I'd recommend a check out in statistical learning theory is basically looking at different model classes and how many different ways they can instantiate that function and you have something that's super kind of constrained like say a linear regression or just a one-dimensional threshold something like this there are relatively few ways you can separate data sets and so it takes fewer examples to learn whereas an arbitrary universal neural network kind of thing potentially has a higher quote-unquote VC dimension it's going to be able to shatter split up many different data sets and it's harder to learn because there's more possible correct implementations so that's a single machine learning model or when you are building machine learning kind of system you are actually learning a function or you're implementing a function that will emit a function like this so we go from having many possible implementations of that predictor to many to the power of many possible implementations of your software for emitting that predictor and so this is just kind one kind of way to bring a lens onto why this is going to be really challenging and in fact it's actually worse than that because you have hyper parameters so there if you ever use any of these libraries of SK learn or similar you're going to see a whole bunch of arguments that have these handy-dandy default values but all of these typically mean something quite reasonable and they're sometimes called hyper parameters and so actually you have to say I have a learning approach and I want to set the hyper parameters and I'm going to output this and that becomes an even blows up the space of possible implementations even further now this is kind of what the Google paper is often talking about it can actually even be much much worse than that so this training data probably is coming to you from the outside world and your predictions hopefully if they're having an impact and driving the product and doing something useful I'm also going to go out there in that outside world and potentially there's going to be kind of feedback loops and things like that right so this this kind of phrase has to do with Wall Street to engine not a camera and the idea of sort of machine learning models you can one sort of school of thought is this just a camera that captures facts and captures regularity about the world and that's that but the other one is that actually they're getting the models as people interact with them and game them are going to drive and change behavior and so anybody who kind of works in this sort of relevancy or ad tech or this kind of thing you know the output of your model the output of your recommendations are going to affect behavior that is going to be used as training data and your further recommendations and this is something else that's really called out in the Google paper so I hope by now kind of convince you this is some non-trivial sort of software engineering challenges we can kind of lay it out a little bit as a map of potential problems and a lot of these again kind of map to the structure or the outline of that Google a high interest credit card debt paper but upstream you probably have a data pipeline that's getting this data from somewhere you can just have straight-up bugs in the data pipeline you can have changes in the data pipeline if it relies on outside data sources that your team or yourself don't control you can just have dirty data incorrectly you can capture it or instrumented stuff then you're going to learn a model from it maybe you picked the wrong model maybe your model has weird failure modes you're going to serve these results now they might not actually make sense to people or the model you chose might have a big gap between what that model emits and what kind of actual end user wants to consume and that's kind of gets into this last phase ultimately for the most part with very few exceptions your customers your end users are not consuming the output of a machine learning model raw you're not doing this tree as a service probably there's some course sort of last mile user experience wrapping and things like that and that's another place where you could have some potential for serious problems and headaches and of course you know there's always the loopback in software engineering machine learning engineering you're always kind of iterating back and forth and this is yet another place where you can have some problems you've changed the features in your serving layer but not your training layer you've reality has actually changed out from under you different behaviors are kind of coming in on your data or your upstream behavior is different or you've just introduced a code regression so this is kind of yet another way that you can sort of have some more problems creep into your machine learning software system and all of this by the way of course assumes that you're sort of base one on one level software actually just doing what you would hope anyway so that's on top of this you have these other challenges so what kind of tools do you have at your disposal all right we're gonna walk through some of the possible portfolio you can kind of build of approaches and solutions to at least limit some of your risk and exposure to these problems so this is a really funny kind of anecdote from the genetics research community and so I don't know if folks have worked in genomics genetics art biomedical informatics but you have all these genes and you have their names and there's sort of these code words all over the place and they often just kind of a jumble of letters and maybe some number and these actually refer to genes and humans or other model organisms and like many people biological scientists using Excel and actually they were having an issue where Excel was automatically suggesting that particular gene names are actually dates step two which is some gene is actually being interpreted as September the second and so this kind of bug you know you may may or may not be yourself working on genetic data but this kind of thing can happen all over the place and actually apparently was happening 20% of the time to various people so one possibility here is actually considered your data pipeline within the scope of test coverage and automated test coverage and this one particular instantiation of this idea is an open source library called create expectations and basically you can thing as a sort of monitoring or unit testing for your data and you would kind of bake this in your data pipeline potentially and assert certain conditions or invariants against this data and it will fight fail like a unit test against your code and so an example test assertion would be something like this and this is from their documentation and they walk through the sort of classic Titanic survivor data set and passenger class and you can basically say this column is going to come in as a string because it's coming from some other pipeline or sequel database or whatever and I expect only ever to see these three particular values these three particular strings in that column and so if you have something that you are considering to be kind of an enum style piece of information but it's coming to you from some upstream place where it's encoded as a string you can kind of encode this sort of set constraint and say I only ever expect this column to take on passenger classes first class second class third class if I see a calendar date if I see somebody's name if I see an empty string I want my pipeline to throw this error and you can kind of run this against your data pipeline as it flows through and there's an entire library of sort of similar assertions I expect this to always be a number I expect this to be non-negative I expect it to be this that or the other thing so in basically building this India pipeline is one way to kind of protect against some of those upstream sort of dirty daily dirty data sort of failure mounts kind of sort of just a bunch of jumble of keywords there's a whole universe of research going on right now it's a very hot kind of academic topic that I'm not going to really have the time to dig all the way deeply into but making the model itself a robust and resilient so you can say I want my data to have this to these certain characteristics I want my software and outputs to have these other characteristics but this research sort of says for given model classes model learners and actually the model itself how can you ensure that the model is not going to have extreme failure it's in the presence of certain kinds of attack vectors or failure mounts and so what I mean by that is you miss specify the model entirely so this might be in the case of a probabilistic graphical model and you've just got the wrong model for the data and you're have dependencies where they shouldn't be or vice versa how badly is the performance or output of that model going to be degraded in the face of you slightly miss specifying choosing the wrong machine learning model another one is data set shift so there's kind of a classic machine learning AI story of DARPA working on this tank detector that would take an image and tell you if there's a tank in it and it seemed to be getting 100% training set accuracy and all this great stuff and then they actually tried to field it and it was auto failure and it turned out that the kinks were always either in front of a forest or it was daytime or something like this right but basically there was a data set shift where the images of the tank from the training set we're always in front of the forest and all the sudden in the test set they're not in front of the forest and of course the just dumb statistical machine learning model did not kind of anticipate that data set shift so that's another kind of degree of robustness to the extent that the real world or the actual data does not exactly come from the same distribution as your training data that's another kind of challenge that your model is going to have to face and how do you bake that into the model itself another one is this sort of adversarial case so this is the notion that you're going to have potentially an adversary who can tweak data in particular ways and cause your model to give very extremely different outputs and the sort of the research papers on this the example they use there's pictures of pandas and pictures of stop signs and because of sort of the nature of how a deep learning model works they're able to inject some very targeted noise and get it to completely fail and so how do you create a model that's a little bit more robust to that and sort of the last kind of classes sort of sensitivity to outliers and this is the case where if you say I want to fit a line to some data if you're doing standard ordinary least-squares and you have one data set where our data point where the data is just kind of arbitrarily fired from the line it can drag your linear fit arbitrarily far in that direction if you're just using sort of a normal squared loss and there's a whole kind of universe of research going back many years on how to do that but basically some updated thinking for that how do you have individual dirty data points or misspecified data points or just straight-up malicious data points not throw off your model in a fundamental way and so there's a whole ton of academic kind of research about that calibrated predictions are another thing that's going to make your actual model a little bit easier to understand and interpret and work with with respect to your rest of your system so this is from the secular Docs and basically the idea of calibration is if I output 30% probability that it's spam and put all of those in a bucket will 30% of them actually be spam right and this is a property of particular modeling classes or post-processing of models that they have outputs that can be interpreted as genuine probabilities as opposed to just a score high spam high score equals spam low score equals not spam can it be interpreted as actual probability this is something else that's going to make it easier to kind of work with and of course just kind of sort of standard software engineering advice is that if this is not your core thing that you're doing if possible you can kind of buy or borrow or open-source instead of building your own there's actually super interesting talk yesterday about this onyx machine learning representation framework and how to serve it from Scala and things like that of course all the cloud providers would love to sell you their kind of mla as a service but if this is not the core kind of thing you're doing this is a great way to avoid shooting yourself in the foot in a ton of different exciting ways so now kind of three T's or reliability potentially you know how can we use these kind of tools types testing and monitoring to kind of enhance the robustness and reliability of the overall machine learning setup this of course can types help again with this audience probably guessed folks kind of inclined to think yes but much like standard software engineering you want to let the compiler catch some of these areas for you so very often in machining pipelines and such you're working is a really trivial example what you're working with time and just passing Long's around your codebase is it a days in a millisecond is it a year it's really easy to have really wild kinds of bugs whereas if you're actually using the sort of proper library support or you kind of constraining baking it in to the datatypes themselves it's gonna be a lot harder to have those kind of problems and again back to that types first test thinking you dramatically sort of constrained the universe of problems that you can cause yourself so one sort of interesting data point in the space of how you can use types serve enhance the reliability or correctness of your machine laying pipe line is the open source Salesforce machine learning library and what they've actually got here is a pretty nice sort of well thought out kind of schema for how to think about different types of features and to the extent that you have this kind of baked into your pipeline through your training your testing your serving your ETL is going to make it a lot harder to sort of introduce bizarre mistakes where you are interpreting x as some other numerical quantity or you're interpreting numeric I quoted number as an actual string or things like that so this kind of forces a little bit thinking and effort upstream like any kind of type based solutions do but at the kind of advantage of avoiding a whole sort of class of interesting failure modes another sort of thing that's its own whole talk that I talked a little bit about last year here at this conference was you can do information flow controller types why this can become relevant in a machine learning or data science con context is you might have certain information that you don't want kind of bleeding or leaking from training and validation to your actual test set or bleeding from the future into the past or all sorts of other ways that information kind of flow through your software system in a way that's going to cause you to kind of fool yourself about how well your machine learning model is actually performing and actually the context that I was interested in when I had sort of talked about this last year was data privacy and sort of sensitive information control often machine learning systems being trained and run against data associated with actual users or customers and to the extent that you don't want individual R&D engineers poking around in that data or that data leaking through your software system out to the outside world in some way you're handling sensitive data you can kind of bake into the type system tell me that this data is sensitive and I don't really want it to kind of go in different places so you can check out that talk if you're interested in that testing so of course we love when computers can do work for us and we'd like to test our machine learning model isn't be pretty sure that they're doing what we want however there's some extra challenges to testing machine learning models often might have non-deterministic or a very unstable behavior on any individual data point so it becomes a little bit harder to unit test this one data example should be X Y or Z also these labeled examples are potentially really expensive and you actually want them in your pipeline being used for training anyways and how can we kind of get around that so property testing this is just unnecessary machine learning thing as such but just general sort of framework for doing automated testing where the computer is going to do something useful for us you just assert that there's a property that my function should satisfy for all inputs so for example I could say I want to do ordinary least-squares for all weights that I learned from any data set X I want to have that it converges to a valid parameter vector okay and a kind of constrained version of this would be that I want for any input data set that has no not a numbers no infinities sort of well-formed data set my ordinary least-squares machine learning training should arrive converge to a realistic output vector so what this actually come means in practice is you can use these fancy property testing libraries they will auto-generate nice test cases you can have combinators for sort of building these test case constructors they will shrink examples down to the minimal failing case do all sorts of nice stuff to help you have automated testing on your data but where this really kind of comes potentially in the machine learning is this metamorphic testing ideas is something really interesting where supposedly we claim the claim of these researchers is you can kind of alleviate the Oracle problem where do I get the label data so based on the name it kind of sounds like something that belongs in this conference but the basic idea is we want to find these metamorphic relations that hold our sequences of inputs outputs the library or the implementation the testing can then automatically transform the inputs and just check that the relation holds over the outputs so you don't actually have the output label you don't actually have the ground truth output but you know that if this input came to this output and I tweaked the input I should expect the output to be tweaked in a certain way a little abstract some specific examples and this is from the Chen survey say that you have a database query adding a disjunctive term should only ever increase the number of results returned so actually the output should be a superset it might not be proper superset it might just be the exact same but adding that disjunctive term should only ever result in more results the shortest path an undirected graph from A to B should be the same as B to a and you know there's all sorts of a huge universe of kind of testing programming language research in this area where you can see kind of what they were able to do with it so pulling it back to machine learning one big challenge again a machine learning is it's really hard to test and verify that your implementations and your systems are kind of doing what you think so in especially in unsupervised machine learning and so this is really cool paper where they are basically saying how do we define metamorphic relations and do interesting testing clustering which is kind of a notoriously difficult to evaluate sub approach of machine learning and just really hand-wavy here clustering you know basically given a pile of data break it into a few different piles that's the clustering problem and what you'd like is that within the piles the points are very similar to each other and the piles themselves are dissimilar to each other so thinking about kind of spatial representation a fixed length feature vector in this case maybe we have all of these little circles and distance function we've Mustard them into these little that sort of correspond to a human interpretation now say you've got a cluster of implementation how can use metamorphic testing to test it right and the authors of this cool metal paper define all of these metamorphic relations that should be expected to hold over clustering a lot of very come standard stuff where if you translate rotate shift these feature vectors they kind of do what you think if we shrink points towards the centroid they should stay assigned to that centroid if you add extra dimensions that are totally redundant with respect to the existing dimensions it shouldn't affect the outcome if you add outlier data points it shouldn't affect the clustering of the entire data points and so on so there's kind of a ton of different ways you can say without having ground truth data you can just say is my clustering algorithm sort of robust or does it behave as I expect in these different ways and so these are the key references again I'll kind of send it out later but there's some really cool stuff there all right managing the lifecycle I guess maybe I'll cover this really quick but everybody uses I'm assuming source control and stuff for your source right and likewise there's a whole universe of tooling from open source and commercial side to basically try to do similar thing with models including some folks are actually here they've done some cool stuff monitoring you want to basically stand up look at these input features and output features and see are they actually behaving like we expect this is kind of a whole universe if the input fields change dramatically you want to know about it because it might cause the way your model behaves to change dramatically as well actually now I have say that you have some individual model output I want to be able to explain it this is another huge area of research one of the most popular papers sort of recently here was lime and if you're going to try to do a literature search on this how can i anchor your search there and look for more recent work that cited line that would be my sort of research hack to find some of the cool new research here but the basic idea is say that your image classifier has classified this image as class X or it's classified this he meant s-class why why how like which fields were the most salient in terms of the model making that prediction and once you're trying to investigate problems with your model your pipeline your data everything those kind of more granular debugging explanations can be super valuable and of course there's a whole bunch of people or countless research open source and commercial as well all right coming up on the end so say that you know people get very unhappy with machine laying systems do not work well and there's actually some really cool psychological research where they are really really unhappy in fact more unhappy than they would be if they thought a human screwed up that way so humans are kind of willing in this kind of psychological research to cut other humans some slack so they'll do better next time they learn from their mistake and so on and so forth whereas the computer screws up you are furious there's no coming back from that and so what can we do about it so they actually did some follow-up research where they determined that if the user is allowed to tweak or tune the output a little bit they've become dramatically more forgiving even if these tweaks are like really really small in scale so the implication is you should give your user some levers or knobs to correct or adjust those outputs and so you know kind of example we actually have on the super logic product we cluster your logs but if you don't like the cluster you can kind of explode it into smaller clusters and this really kind of helps people feel better about it so I'm gonna run out of time so a little quick there's one kind of case in which sort of an academic machine learning yeah the incentive is to kind of pull more and more of the problem into the model and when you actually trying to build an industrial system this may or may not be a great idea you know actually you might be better off thinking of your model as a small piece of a more isolated or a small isolated piece of a more coherent larger system and you could potentially sort of fix the mistakes or post-process your model in different ways to kind of get a better shot at it and you might actually want to sort of enforce sanity checks or things like this you can add some dummy knowledge put wrapping logic around your model to avoid these kind of mistakes all right the very last thing a legend of the Russian space pen we're not to use learning this is a fun story not actually true so the problem is that your pen doesn't work in space your gravity so NASA spends a billion dollars and develops a space-age super pen and Russia uses a pencil and the you know the middle listen here why it should be true even if it's not true story is you can always redefine the problem and this is especially a big problem in machine learning because machine learning data science people are always looking to phrase problems as machine learning problems even if they are not necessarily problems because sometimes it really pays off so this is a really cool database paper from Google where they take previously a not machine learning problem and databases turning into machine learning problem and this is a thing that people think is a face on Mars right or the moon and they actually I forget if it's march to the moon but the problem is that has a machine lying person you see machine million problem everywhere so say that I have latent users Netflix I want to mix your profile very cool machine learning profile problem there's three people in a family they're all using Netflix who is viewing what let's do it with machine learning no Netflix just asked who's watching in machine in like operations is seem illogical example people want alarms to not go off for their system when you're having regular maintenance can we auto learn the maintenance can we detect the seasonality to a fast Fourier transform do some an econometrics or to seasonality prediction or more likely probably what you want to do is to create an API that just lets the user say that maintenance is happening and they can put it at the front of their maintenance script right and so one last kind of thing the best machine learning that you can ever have is when you don't have to do the machine learning so despite kind of your interest in skills I would definitely encourage you to also think about it that way is way where's the Russian Space Pen in any given problem right so final takeaway very boring exercising eat your vegetable there's new failure modes and machine learning there's still no silver silver bullets hopefully I can give you a survey and overview of how you can build a portfolio of approaches to kind of get a more reliable and system out of machine learning so thank you very much [Music]