Scale By The Bay 2019: Rob Monro, Discovering Your Model's Known Unknowns and Unknown Unknowns
[Music] all right for those just coming in I think there's still a couple of seats vacant at the front here so you should be able to find somewhere to sit so thanks everybody for coming to this session so everything that I'm presenting here today is taken from a book that I'm currently writing human-in-the-loop machine learning by men in publications as with all many publications these are the book is being published chapter by chapter as it's written there are three chapters four chapters out right now three more than Mediate pipeline and and four more to come early next year this is my pleasure to really give you all the best bits of the book so you don't need to buy it here today but if you do I'll also have a code for a 50% discount at the end of end of the day so my talk today is about discovering your models known unknowns and unknown unknowns so how do you know what's missing from your models information and what your model knows that it is already uncertain about so I find it useful to think about machine learning in terms of a knowledge quadrant what are your known knowns your known unknowns your unknown knowns and your unknown unknowns so you're known owns are everything that your machine learning model is currently confident about this is what your model can do today its confident that it's getting the answer right whether that's predicting a label or generating text or identified an object in an image especially known knowns your known unknowns are the things that your model knows that it doesn't know so this is when for example if you have a binary classification task and the confidence is around 50% this tells you that your model is very uncertain and so these are the kinds of problems that you can address with uncertainty sampling your unknown unknowns the things that your model can't currently do I mean might be uncertain or it might you might have a very confident model prediction but it's completely wrong because it's from a domain of data that your model hasn't seen before needless to say it's a lot harder to identify your unknown unknowns and so these are what your address with diversity sampling and I'll be giving examples of uncertainty sampling and diversity sampling so it's not part of today's talk to fill out the quadrant we have transfer learning so taking existing knowledge which is captured in some third party model or system and incorporating that so this is your your unknown knowns so this is latent information that already exists not currently in your your model but you know where to get it elsewhere and where this fits into a a typical machine learning human-in-the-loop cycle is up here for a proactive learning so you have a deployed machine learning model you're able to sample your new unlabeled data items and then you only have a limited budget in terms of how many of these items can be put in front of a human for review maybe they're in-house workers maybe using crowd-sourced workers or contract workers regardless of what workforce a combination of workforces you probably don't have the budget to label every item in fact almost certainly not if you had the budget to label every item you wouldn't need to do which he learned it in the first place you could just be a a human task so my book covers every part of this cycle this talk today is focusing on active learning so what are these most important unlabeled items that we can put in front of humans for review to make sure that our model is as accurate as possible and is able to adapt to the domain that we're applying our model to especially in cases where we know that our data is changing over time so first of all an uncertainty sampling so this is where your model is confused how to identify the different kinds of confusion that your model might have and so that we can focus on slightly more scientific looking picture than the robot items which tend to be near your decision boundary now so here we have a two class problem label a and label B we have unlabeled items and uncertainty sampling should focus on items which are near the decision boundary so unless you only have a very simple binary classification task there are many different ways that you can calculate the uncertainty so I'll go through all of these here I think these are really important equations the examples that I'll show I'll also share the PI torch code for implementing them all they're all only a few lines of code long and I think it's a real shame that most people coming out of data science programs whether it's an online course or even college they don't get taught these algorithms you get taught all kinds of architectures for a building machine learning algorithms we don't get to any of the algorithm any of the equations for interpreting that model confidence so pie charts examples would be applied to the output of your machine learning algorithm which looks something like this so this is your probability distribution we're assuming it's a full class problem here you have different confidences of each of these labels so the second one here is obviously the most confident and so this could be a PI torch tensor like the one that's declared here in the code so this is the type of output that you will see with softmax just as an aside softmax is a terrible way of generating confidences it's it's good for optimizing against especially if you if you have your labels as ones and zeros that you can optimize against with a gradient descent or similar function but I won't go into the strengths and weaknesses of sofmap zero so let's assume for this tour we already trust our confidence distributions so the first and most obvious way to measure confidence is called least confidence this is simply looking at the confidence of your most comfortably predicted item so whatever your label was predicted to be so in this case it was the the second of the four labels was the most confident and we're looking at the difference between that and 100% confident the rest of the equation here is just making sure that this becomes a zero to one value where zero is the most confident and your most uncertainty is at one it's really important when you're deploying these algorithms that you're really consistent some of the stats are positive some are negative some abounded some are unbounded if you're trying to incorporate multiple ways of measuring uncertainty into a downstream task you're gonna get into trouble really really quickly if it's not in a 0 to 1 range or some other consistent range across all your metrics so this is something that I really highly recommend if you look at the research papers they're not normalizing because they don't need to but I think it's something that's really important here so marginal confidence is the the next most simple way to implement uncertainty sampling so this looks at the the difference between the most confident prediction and the next most confident prediction so again simply scaled so that it can be on a zero to one range so here is the difference between the second and the fourth labels very similar to that is ratio of confidence but rather than it being the difference between them it's the ratio between them so it is the their second most divided by the most confident prediction that you have if you're used to working in with log probabilities which is probably your go-to if you're doing a lot of machine learning so you don't to worry about Devon flow issues that you get with regular probabilities then this is just the difference between your L log probabilities because obviously the difference in log face becomes the ratio in natural numbers finally entropy is a way of measuring uncertainty that takes into account all the labels that you're predicting so your measure of entropy is a measure of how close these are to the null prediction of the mall having exactly the same level of uncertainty so the other measures only took into account either the most confident item or the two most confident items this takes into account all the predictions so to get a feel for what this looks like on real data let's assume that we have a three class problem on the on the Left here that's your left yep as each of these black dots represents a different class so you can see that margin of confidence and ratio of confidence because these are the pairwise measures have these long radial arms anything that's pairwise confusion is where they emphasize the most whereas entropy because it's looking at all of the predictions the biggest places for confusion right in the center of it all so that's where you get the largest confusion between all three and if we scatter these four are for something less uniform on the right then you can see how you really are sampling from very different parts of even just this this two-dimensional space with the different kind of algorithms so these graphics came from heat maps that I just created quickly in JavaScript you throw up my website under if you can read that link there I can tweet it out later you can play around with it click and come up with pretty heat maps yourself of these different algorithms I find it to be a really useful way to get an intuition for the different kinds of uncertainty algorithms and come up with the decision about which is the best one to implement for uncertainty for your particular data finally some of the more advanced so called Bayesian deep learning approaches to uncertainty introduce noise into the model itself and try to get multiple predictions so if you've ever build machine learning models I'm sure you've tried building models with dropouts you're introducing randomly dropped out typically neurons sometimes just edges as well as you're building the model and that noise helps you have a more robust model especially if you're a PI torch user you probably accidentally kept your dropouts in during prediction as well and and send in accurate results so actually you didn't make a mistake you're using cutting-edge answer and he's sampling that's exactly what we're doing so you get an item you do multiple predictions with different random dropouts with those predictions and if you believe a dissertation from ten years ago and assume that you have infinite neurons that range of probabilities you get will actually be a Gaussian curve I think the math is a little shaky but because you will get a Gaussian curve of distributions that's why it's known as Bayesian tape learning this is a really reliable way of getting confidences from your model if for example you don't trust softmax or you have some other output function for which isn't generating a probability distribution and so it's really only be in the last couple of years that we've started to see researchers use this method for getting confidence from their models I haven't seen it yet be deployed in the industry but I'm sure it will soon okay so that's uncertainty sampling there are different ways of knowing what your model doesn't know and so here were something two building blocks for the the somewhat hotter task of diversity sampling trying to understand what your model does not yet no gaps in your models current knowledge so diversity comes in many forms it may just be things related to fairly innocuous rural differences like different accents it could be changes over time a lot of the time you'll find that diversity and potential bias with the new data will correlate with real-world diversity in ways you might not have been anticipated so the recipe sampling is really important if you want to make sure that everyone who is using your model is able to do so equally so in contrast to uncertainty sampling where will model in try to sample items near the decision boundary diversity sampling looks for items which are not like anything that has been labeled in the past or if you used in diversity sampling to begin with looking to maximize the diversity between items so notice that these five items which have been sampled for review they're not similar they're not anywhere near the existing labels and they're also not near to each other and so that the goal of diversity sampling is to cover as much space as possible and you come from the the reinforcement learning school you've probably heard this code exploration versus exploitation in terms of looking at model uncertainty or looking at new areas I don't like that analogy because I can never remember which one's exploration which one's exploitation I always get them backwards I think they're yeah that means they're fun they'll iterate in their rhyme I think it's a useful analogy if you do come from from that background so two methods which are really commonly used for a diversity sampling mobile-based outliers and cluster based sampling so one of the things that you lose in the final layer of your model if using softmax for the output is any indication about the level of activation in your model so softmax is only sensitive to the relative differences so if your logics going into the softmax function are one two three four you're going to get exactly the same result of confidences as if these were 100 100 200 3 104 and obviously that's just very different in terms of like your model activation one's very highly activated ones fairly low early activated or you might even have negative activation if you're not using a rel that clips at zero so model based outliers let you find this information again so you look at your logics your logics you also look at your hidden layers and you look at the overall rate of activation in those layers what's a little bit tricky is that the average amount of activation in any one neuron is completely arbitrary you're comparing apples and oranges if you're comparing different neurons and so in order to establish what does count as being low activation in a given neuron you want to first calibrate this with held-out validation data from the same distribution as your training data so this allows you to turn each activation and every neuron into a quantized score which means that you can now look across your entire model and say okay this island I just predicted sure is high confidence but it barely activated any of the neurons in my model this tells me that it contains new features that I haven't seen before haven't seen before in these combinations so something new that should go in front of a human for review so that's the principle behind model-based outliers a lot of the more advanced techniques in selecting items Directive learning rely on this same principle of looking at the properties of your model itself I won't go to those in the talk today they're in a chapter in the book I've just written which is not yet published but if this is interesting for you and looking at the properties and information held within a model then I encourage you to look into this kind of research by far the most common method and the first one you should start with for a diversity sampling is clustering so using unsupervised learning to cluster your data and then sampling items which belong either to the centroid or just randomly selected from different clusters you can also do some outlier detection here as well so do clustering and then find items that are not like any of the clusters that you've seen before potentially to find new items this is an incredibly effective way especially in combination with some of the other techniques one of the problems of uncertainty sampling is that you'll often just sample items one very specific confusing part of your feature space so if you can combine cluster an uncertainty sampling you can ensure you have a diverse set of items which are also very entered for your model there are many different algorithms for clustering that you can use I've never seen any evidence that a sophisticated and expensive clustering algorithm results in better samples for active learning than a very simple one you can use Luke Lydian distance or cosine distance for k-means and i really haven't seen any evidence that other kinds of clustering algorithms might be better for active learning certainly though this is a active research area and I'd love for someone show otherwise another really common form of diversity sampling is representative sampling as a representative sampling is when you know that your training data is from a different distribution than from where you want to apply your model so maybe you only had some data or annotate it to begin with oh maybe your data is changing over time this new data comes in so if you're classifying news articles and the new topics in the news so representative sampling similar to clustering or a built in any kind of model you build a model and your training data you build a separate model on an item sample randomly from your application domain and then you use that to identify the items which are most like your application to main relative to your current training data so this is a very effective way of finding those items which are able to as quick as possible adapt your domain adapt your algorithms to new domains if you look in the literature there is a lot of papers about domain AB dictation without additional human labels it's trying to adapt these algorithms without any human input these are really interesting but I've never seen them used like that in industry when anyone's trying to adapt models in industry there's always some level of additional human verification or or labeling in order to get that that accurate adaptation finally sampling for real-world diversity is something that brings together every other kind of active learning strategy so one of the the most fundamental ways that you can ensure that you have diverse data is to label your data for the different demographics that you care about and look at the different ways that it might be over-represented or underrepresented so it might be a little bit small up in this screen here you can see that as some of these demographics here are well represented there they've all been labeled some others have not been labeled at all some have been labeled partially but only one part of the feature space so they haven't been labeled in a representative way and whether or not this metal actually depend on your algorithm some algorithms don't need your training data to be representative of the real world in order to be fair other algorithms do and that representative verses represent a distinction is something I go into more detail in my book so something like I highly recommend is that if you find unknowns in unknown unknowns and other people's algorithms that you report them so I'm not sure if any of you saw early this week those Nautica the New York Times about bias in Beart and other pre-trade models so this was based off a blog post that I published on just this Monday where I found that for possessive pronouns like here's an her as contextual models like bird preferred his over hers in 99% of cases showing an incredibly bad gender bias and then when I was trying to track the the source of this and come up with solutions to it I found that most of the major syntactic pauses including Google's and Amazons and Stanford's do to recognize hers as a pronoun at all it thought it was a noun or a adjective so I reported this six weeks ago now to those systems they hadn't updated them as quickly as I would like and so that's why we ended up publishing an article on it early this week and yeah it's good to now hear that all three of those are actively working to correct the biases in their systems just as a final note these all the building blocks our show today are exactly that there are building blocks if you're implementing active learning you should think about these building blocks but then how to use them in combination so that you can optimize both uncertainty and diversity for your algorithms all right thank you I think I probably have two minutes for questions something like that oh you've got about five okay wonderful oh and read that ts loop code will get you 50% off my book at mending books today are there any questions no questions oh there we go great darkness are you planning to combine this approach with some of the auto ml methods yeah so the question was am i trying to buy this some of the auto ml methods yeah very much so a lot of the auto ml methods are baking in a lot of these algorithms right now but I think especially uncertainty sampling ones although I I would argue that as a data scientist you should still know what these algorithms are doing behind the scenes just as you do for the the model building components and yeah I've released all these methods in PI torch published on the PI torch blog about that so I'm hoping that some of the open source rather than commercial auto ml frameworks will be able to integrate with a pretty seamless way great talker I'm looking forward to the radio book how expensive are these methods if you're gonna use humans in the loop how expensive using computational expenses yes incredibly cheap so these like there's there's not even any loops here right well there are actually just over the array of outputs so like fractions of a second and especially when you're thinking about this in terms of sampling items for human to review there's really not much cost cost at all the only thing that can take a little bit of time is we're going to get a very large number of predictions more accurate Monte Carlo or some clustering algorithms can be incredibly expensive but like I said there's no evidence that even though they create better clusters in terms of information theory that they're better for active learning sampling are there any other questions hi I had a question about the real-world diversity related sampling yeah so does this imply that you have to know the demographics of where your training data is coming from before you can do it in order to like how do you identify where your knowledge gaps are right yeah so you there's different kinds of demographics you might look at so if you already have that demographic information that's great if not then one things I encourage in my book is to annotate for demographics as well if you're ready having humans annotate for your labels you should be able to annotate for demographics as well sometimes you could pull that automatically from the data as well so when I found this bias it's because there's just like a limited number of pronouns in English and so just programmatically I could look for how accurate each of those were any last questions already perfect all right thank you [Applause] [Music]