Attention and Memory in Deep Learning Ne...

ai.bythebay.io: Stephen Merity, Attention and Memory in Deep Learning Networks

Recording: ai.bythebay.io: Stephen Merity, Attention and Memory in Deep Learning Networks

[Music] [Applause] Well, thanks guys for having me. Um, now as I mentioned, I'm going to be talking about attention and memory and deep learning. But with that having been said, uh, as with all the speakers you're probably going to hear and have heard already, I'm going to give a few disclaimers. So, whilst I like deep learning, I spent an awful lot of time ranting about it. Um, there's a lot of AI hype. There's a lot of uh times when the tool just doesn't really fit the requirement of what you've got. Um, so you can go and read these in your own time. Highly recommended, uh, even if I'm a bit biased myself

But the main thing for me is deep learning is a bit like a jackhammer. and it's uh difficult to use. Hyperparameters will be the bane of your existence. The size of your uh uh model, whether or not you apply dropout, where you apply dropout, all these things will come into play and they'll have horrible impacts on your model. Additionally, it's expensive and slow. You're going to need GPU, at least one, if not many. And they're all horribly, horribly expensive. And on top of that, as Eric Jang recently mentioned, uh, as technology advances, the time to train a deep neural network seems to remain constant

Um, this is like one of those typical things where no matter how much computing power you have, your life doesn't really actually get a whole lot easier. And finally, it's usually the wrong tool for a lot of situations. So, I I highly recommend, you know, taking a step back. It's good to get familiar with traditional machine learning techniques anyway because a lot of those um techniques will actually come into play even if you realize in the end that deep learning is a you know potential solution for your problem. But no one ever gets points for you using a jackhammer for trying to make scrambled eggs, right? I'm not going to give you any points for that. And that's what a lot of these people are ending up doing. They'll throw a GPU at a problem where logistic regression or random forest things like that would actually be the optimal way to go about a problem. Having said that, it can be really useful when you find the right task

Usually one that involves, you know, extracting information and producing higher representations or ones in which, you know, maybe the the feature set is just not something you can really contend with easily. In those situations, this can be the right tool for the job. Uh, and that's really why, you know, I I do the research that I do. So the key idea that I'm going to repeat today over and over is whenever you see a neural network architecture, you should ask yourself how hard is is it to pass information from one end of it to the other. And we can think of this in terms of information bottlenecks. Um there are a number of different types of bottlenecks. One is just the capacity of your neural network. Usually you can't represent all of the features that you're passing in

It doesn't make sense. In the case of an image, for example, you know, we're talking about hundreds of thousands of values. um or in the case of bottlenecks when it comes to gradients. So gradients are the things that dictate how we update our model, how we improve the parameters to better fit the task at hand. And a lot of the time you'll end up with these bottlenecks in between that actually prevent your model from learning properly. So both of these things are huge issues you need to contend with. And a lot of the recent advances in the state-of-the-art things like residual networks or going from a recurrent neural network to a long short-term memory cell are all thinking about these information bottl and how to get around them. So as far as memory itself is concerned neural networks already have this memory by default and that's a hidden state

So after each and every kind of operation matrix multiplication and activation you can consider that as you know the next stage in the memory of your neural network. Um and you know usually you have a number of these layers and after each and every step it's updating uh you know computation and updating its internal state and it's obvious if you you know decrease the capacity of one of these layers you actually have to discard information really obvious but I mean it's something that's useful to point out because it's not always a problem but you need to know exactly when you're doing it and whether or not it's the best choice at that point of time. So a good example of this is if we're classifying you know digits in emnest for example the earliest input you get is all of the raw pixel values. So grayscale values and there's 28 x 28 of them or 784. And if we have a three layer neural network the first layer is you know the input where we read in these pixels. The middle hidden layer might end up being something that ends up recognizing the strokes. So it ends up recognizing higher level abstractions and the last layer is the digits. And whilst we are discarding information, it's not information we generally need

If we're talking about stroke types, you know, we've got a diagonal slash and it might pick that out. The fact that, you know, we don't actually have the original pixel values anymore isn't really a problem for us. That's not the task that we're interested in and that's not the information we actually need to answer the problem at hand. So the information is lost, but the relevant information we need for our question has been extracted. So in this case, you know, short, not a problem that we end up discarding this information along the way. And in fact, there are cases where compression can be highly desirable. So a good traditional example that you're probably all familiar with is word vectors. But think of it in terms of information or information bottlenecks

Um if we're training a you know word vector um by using context on either side of the word, just one word, we end up having the entire potential vocabulary on the left side. So in this case it ends up being the word new and the entire potential vocabulary on the right side in this case it ends up being city and from that context of new something city we want to guess York and the way which we end up doing this is we could have a huge hidden state in the middle and that would you know enable you to kind of memorize a lot of these different contexts but instead we actually force a model to discard a lot of this information. we force it to have this tiny hidden state and when we do we actually mean that it forces the model itself to learn to map these similar words to similar representations. It's essentially finding its own way to cheat through this problem. And that means that you know for the context that we're interested in which is finding similar words and showing that um a certain aspect of similarity or or uh I guess conceptual information is carried through. That's where we end up with that. So color and color being the Australian, British and American uh versions of it and also for things with similar partial usages such as you know say a queen or a princess when it comes to royalty or crowns um they end up being captured by this model because of this force bottleneck. However, it isn't always a good thing

So imagine I gave you an article or an image and I asked you to memorize that and then I took it away from you. If I then started quizzing you on it, sure you might be able to capture a number of different concepts, but I could also, you know, eventually exhaust your knowledge or I could decide on specific questions which are just really difficult. You know, was it a comma or a semic semicolon in a certain paragraph? So the issue is you'd likely fail and even though every one of you is smart, the issue isn't intelligence, it's, you know, working memory. You can't store everything in working memory and neither can your neural networks. Um, so you need to be very kind of cognizant of that as you're designing these things. And additionally, when it comes to question answering tasks, the question is a hugely important piece of context. You actually need to kind of map in to your processing as early as possible because without it, you end up wasting a lot of time and also memory on unimportant details that don't end up serving the uh tasks you're actually interested in. So we can also consider this again uh in terms of the difference between just general vision classification and visual question answering

So in this case what's the main object? It's pretty clear it's a smoothie of some kind and a lot of neural network models are incredibly good at this. But then let's take one step along and say it's a question answering task. And there's actually a huge amount of information that's contained in this image. Most of which is not going to be useful for you at all depending on the question. And it gets even worse as we kind of step into other complex examples. So this image of, you know, a landscape, we have a huge number of potential things that we could end up being end up looking at and a huge number of potential questions that we could be asking could be, you know, is there a train in the image? Which direction is it heading? You can't really tell that here anyway, but you know, are there uh is is a bridge meant for cars or pedestrians? Is there a river under the bridge? And so on. All these things would be kind of impossible to encode into this tiny fixed um hidden capacity that we usually end up giving ourselves. So before I get on to the next section, it's worth just making sure you know everyone's kind of familiar with the concept of recurrent neural networks

And the idea is you can just think of it as a function. And the function takes two different things. It's a new input state. Uh it might be a character or a word, a word embedding. And it's also the uh hidden state from the previous time step. So all the information that we've captured up until that stage, the RNN then basically takes these two pieces of information and updates its own internal state H uh again according to that input and then it spits you out two different things usually an output and its internal hidden state or sometimes they can be the same. In this case we're just going to say h is both the output and also the next kind of step of its hidden state. And this is uh you know basically the simple version of the GU or the LSTM that many of you have heard of

Uh if you you know run into them just think of them as improved variant of the RNN but I highly recommend checking up on the differences. Uh they speak to uh you know our discussion of um kind of residual networks these other building blocks which improve a whole bunch of other things as far as information flow in your model um and are useful in a wide variety of different situations. Now having said that a simple recommendation may be like you know if we end up running out of uh hidden state capacity why don't we just increase this hidden state it's logical it makes sense and for a lot of tasks that's a correct answer but the issue is that the parameters increase quadratically with h we end up with these huge weight matrices that just keep growing each and every time you add just one extra number to your hidden state. So that's one huge problem. I mean training these things is already slow and exhaustive anyway. But the second is that even if we do that the increased hidden state's usually more prone to overfitting. Um especially for recurrent neural networks where you know dropout is usually the way we prevent a lot of this overfitting but dropout traditionally hasn't really been able to be applied well to these recurrent neural networks. So one good way of uh you know I guess contending with memory is not needing more of it or using what you have um better and that's the idea behind variational dropout and these other different types of recurrent dropout that you can apply to these recurrent neural networks

Um I usually refer to it informally as a lock dropout and that's because as opposed to naive drop on dropout on the right uh left where you can see each and every time step you have a different mask um blocking out certain bits of information. Um on the right you can see that whenever you end up hitting the same connection you always use the same dropout mask. So whilst you are dropping information it's always the same information at the same spot each and every time. And that's important because uh on the left if you see the dashed lines if we actually apply this uh you know naive dropout to the hidden state of the RNN you end up knocking out huge chunks of its memory. Imagine you were dropping out 50% of the hidden state then at each and every step maybe two or three steps along most of your hidden state most of memory is all gone. So it doesn't make any sense to actually do that. Um and so that's why a variational lockout uh dropout uh is kind of the good way of rethinking about traditional dropout and it gives huge improvements when it comes to standard RNNs or standard LSTMs because it simplifies the prevention of overfitting immensely and it also ensures that you know generally you don't need as much data generally your models will also fit you know as quickly um and overall a lot of these problems end up going away. A second type of recurrent dropout is oh so I might just mention one thing about uh gals dropout uh which is that it does end up knocking out part of your hidden state

So going back to the question of information bottlenecks because we are dropping out say if we're using 50% we're dropping out 50% of the hidden state we are losing 50% of the capacity that we have in that hidden state. So it's not the optimal solution. we aren't really using utilizing our hidden state capacity as much as we could and this is where we can introduce zone out. So zone out um you can imagine it is kind of like a faulty update mechanism and so you know it stoastically forces some of the current units in H to maintain their previous values. So a good way as I mentioned is just this faulty update mechanism where your RNN cell it goes through one step it goes to try to update its previous hidden state and sometimes it fails and when it does do that it ends up leaving some of the previous hidden state in place but that means you aren't actually losing any hidden state from say earlier in your computation you're just losing updates to it later on so you're still using the full capacity of your um RNN's hidden state. Additionally, zone out can be considered related to stocastic depth if you're aware of it. Uh if not, it's basically just a technique used in convolutional neural networks. And the idea is that you know occasionally you just want to continue information all the way from the start of the input to the end without you know blocking it in any possible way

And both of these are hugely um useful and really easy to implement. And even if you aren't really up for implementing it yourself, they're already part of a number of different frameworks by default. As an example, uh it's a oneline change in caris uh for variational dropout. And it's, you know, something you don't really need to worry about at all. It's literally adding 0.2 or, you know, whatever your dropout number is to your model. So to come back to our original example and then look further into how memory and attention can actually be used for these things, um I'm going to use translation as an example. So you can just throw say English as input and then German as the expected output and your model would will actually learn how to translate between these two different things. Uh after each time step we end up updating the hidden state to contain the encoding the sentence up until that stage

Um and so that's our hidden state the red boxes at the bottom and that contains you know after the first hidden state he second hidden state it contains loved and so on. Now, the big issue with this is that we're trying to force the entire representation of the sentence to live in one tiny hidden state in the middle between this encoding RNN and this decoding RNN. And that sole hidden state might be, you know, 300 um units in size and you're trying to encode an entire sentence into it. You're obviously going to have issues with longer sentences or more complex sentences. And given that we really need to find some way of avoiding this problem to really kind of hammer it home, an example sentence which you know is pretty standard when it comes to a lot of you know say newswire articles uh is this kind of experience is part of Disney's efforts to extend the lifetime of its series and build new relationships with audiences via digital platforms that are becoming ever more important. He added, "A bit of a mouthful, but I mean, you know, you reading it, you don't have a tr a problem understanding it at all." And that's only 38 words. Uh words and newswire articles can get a whole lot uh sentences in newswire articles can get a lot longer on top of that. Yet, even though it's only 38 words, the accuracy as far as translation is concerned drops off incredibly rapidly, even when we're going past the 10 to 20 word mark

So, we really have to try to avoid that in some sort of way. Now, an obvious way to think about it is, you know, human beings translate this stuff all the time. They're good at it. How can we actually take advantage of the same thing? And that's translating one part of the sentence at a time and then referring back to the original source when we actually think it's useful. So, we can emulate this with uh neural networks by providing an attention mechanism over that original sentence. And that allows for longer sentences this kind of shortcut for information to travel across. Not only is the path shorter which generally makes um gradient flow issues uh easier to contend with. So your model gets the you know better supervision as to when it should be using um certain parts of the model or what it should change

Additionally it's not constrained the information from a single hidden state. it can use the output of all the hidden states that it ended up um producing during the encoding step and select individual parts of it depending on what it's actually interested in. So let's kind of go through this idea in general detail. Um for each of these hidden states, we want to produce some sort of attention, right? We want an idea of how important it is to the particular task at hand. And so for each of these hidden states, H, we produce an attention score A. Um and we ensure that all of these A's when you sum them up end up summing to one. And from this we can then produce what we refer to as a context vector. So some uh reflection of the information that we want to extract in the original sentence

And that's just simply a weighted summation the of the hidden states. You know mix them together in some combination um such that you know in total it sums to one. If all of your focus is on a single uh hidden state then you end up getting that hidden state. Maybe it's uh spread across a number of hidden states that are related and you end up with some semblance of the information from all of them. So you can see where that's happening in the model here. Um and so that's literally summation step and it goes across all of the hidden states of your encoder. Additionally, how do we actually ensure that our attention scores sum to one? Um many of you will probably have already seen it and that's simply using the softmax function. It ends up kind of normalizing all of your scores

Um, and it's completely standard in all the toolkits that you're likely to end up going to use. And finally, how do we actually produce these raw attention scores? So, you know, the the ones that sum to one, great. You know, that's what I want. But, you know, it's kind of hard to do that initially. So, you can produce these raw attention scores with a number of different options. Um, one of which is an inner product between the query and the hidden state. It's literally getting the two hidden states, lining them up next to each other, walking across, multiplying one number by the other, and then adding them. Very simple, um, very intuitive, and it's really quite efficient to do

Uh, the other option is to use a feed forward neural network of some kind, one or many layers, and use that, um, basically combining the query in the hidden state and using that to decide whether or not it's considered interesting for the question at hand. Now which one you decide to use it really depends on the task um on your particular um interest as well but most of the time it really doesn't matter strongly. So as kind of a a review of what it ends up looking like we have the attention score calculations in red. So basically pass in some query and in this case it's a representation of the last decoding state. So in this case it's you know what ended up producing air in um German and we end up passing that back over across it interacts with all the uh output states hidden states from the encoding model and then we end up with some weighted summation of those as a context as a new information we bring into the later steps and we can end up actually visualizing what this attention ends up looking like. So between say French and English uh in this sentence there's a lot of one to one uh you know mapping basically as we go straight across you can stray straight down along the diagonal most of the words end up aligning pretty cleanly to their other um language but there is one section which is quite interesting which is the European economic area or zone economic European um and in that case it's actually inverted the war order um between one language and the other and you can see very distinctly that it's recognized that and it's attending to the correct information when it's requ it it really kind of doesn't matter how long the sentence ends up being it still ends up capturing and kind of maintaining the same general state um of of accuracy as well. Now, this isn't the only thing that we can um use this kind of attention for, but it gives a great example of why we're interested in it. When we know that what question is being asked, we only need to extract the relevant information

And in this case, you know, for machine translation, it's really an implicit question. It's, you know, what word do we think is coming up next? And what information from the original source sentence can we use that might better indicate that? And because we can actually use attention, we get a few different advantages. We don't need to compute or store unnecessary information. We don't need to worry about trying to take this entire sentence and cramming it into some tiny hidden state that we end up having this having to pass through this tiny information bottleneck. And we additionally have this shortcut so we can refer back to the original sentence and we don't need to worry about you know leaving essentially information um unreovered. We don't need to worry about losing all that in that transition state. So if you're interested in I guess how neural machine translation ends up playing out when you go all the way to production um I highly recommend my blog post um where we basically go from the same simple model that we just looked at and we keep stepping along. We build up all the way from uh you know basically beginning sequence to sequence learning to uh Google's neural machine translation architecture that they recently published um and show basically they add more depth pretty trivial and also residual connections um which is again a way of you know providing this shortcut for information to travel across and all that's uh at my website

So having seen that uh what are some examples of attention? We've seen it for machine translation but where else has it been successfully applied and what else can we do with it? Um I can't go through all the different things that you can do but one of the simplest extensions is allowing multiple passes over the data depending on how the question evolves. We can actually look at the different sections of input which might you know lead our model to realize that it actually needs to be asking a different question. uh and a good example of that is question answering using the dynamic memory network. Um so the idea here is that the dynamic memory network uh was a way for having kind of modular architecture for a number of different tasks. So the idea we have is we have an abstract kind of input module. In one case it attends to text in the other case it attends to an image and we have a question module for encoding our question in some way that we can end up using in the model. And then episodic memory which allows us to look over our input repeatedly. And this is work from my colleagues Kumar Adall and Jeong and Social

And uh basically it was a number of us all working on this type of architecture trying to work out the best way for this information to kind of propagate forward. So the input module is really pretty generic. Um rather than using a hidden state for each word as we just saw in machine translation, we have a hidden state for each sentence. in the case of text. Um, which we then have another recurrent neural network run over the top of. And for images, we end up basically breaking the image down into sections in image patches. And then again, we run a recurrent neural network over each of those image patches so that it gets some idea of, you know, what information what image uh patches next to me and whether or not maybe that's the same object. both things these things are pretty generic and all they end up producing at the end is kind of a set of output um what we refer to as facts um the same as before as our hidden states so that's what we end up doing our attention over and once we've got that we can basically perform multiple passes so some tasks actually require us to perform multiple passes over memory for actually calculating a solution if I provided an image and I said you know who's the third person um on the left uh first you need to probably identify where the people are and also then work out where the third person is and then you know maybe identify some piece of um identifying information about them and it's the same thing that we've got here

We have the attention mechanism pass over the input using some question that we've potentially supplied. Then it updates its internal memory state and on the second pass it might have actually changed the question that it's asking. Um, so you don't actually need to supervise what the questions are when you get further and further into this hierarchy. It's hopefully learned as part of the model itself. So we can uh actually visualize what this ends up looking like again. So what is the main color on the bus? Blue. Um, and you can see it's just highlighted the sections of the image that a have the bus and b actually contain the blue paint. Same for what type of trees are in the background

you you end up with the answer of pine and it ends up completely ignoring you know the uh C plane in front of it which in most other tasks imagine if you didn't have the question beforehand you probably pay a lot of information to the plane considering it's something that you consider quite interesting additionally the same sort of thing how many pink flags are there is this in the wild that highlights just the man-made structures and so on and all these things are um basically created just from processing the input data over and over and over processing our data set none of of these are hand defined features. the same sort of thing. Uh what is a boy holding? First, it needs to identify where the boy is in the image and then it identifies, you know, where I guess the hands are and it then decides, okay, cool. That's a surfboard. And finally, one of the ones which I absolutely love because I would not have expected the data set at all to have this kind of fidelity information. What is a pattern on the cat's fur on its tail? And it correctly answers stripes. Now, one thing is you could say, you know, a lot of the time neural networks, they cheat in wonderful ways. And one thing it could have done is just looked at a huge amount of text and said, okay, most of the time when you photo a cat's tail, it's striped or it's, you know, patterned or something like that

But in this image, you can see exactly where it's highlighting. So, at the very least, it knows where the cat's tail is. Um, so you can actually kind of introspect the image and see what information it can pull out of it. And the same for did the player hit the ball? You know, it's a pretty easy question. You can say yes or no. But you can see very very very lightly in the in the top right of that image, it's highlighted just the ball. It's midair. So, it's able to actually see that information, extract it, and you can kind of go in and double check, verify that this model is doing the right thing

Another really interesting um use of attention is something called an int a pointer network. And the concept is incredibly trivial. It's almost, you know, childish uh in simplicity. And the idea is what if you're in a foreign country and you've got a menu in front of you and you want to order dinner, but you have no clue how to pronounce anything. The easiest thing you can do is literally just point to the the image on the menu. Uh, and this is a motivating idea behind, you know, pointer networks. Um, pointer networks help tackle the out of vocabulary problem. And that's a huge issue because most models have a pretty limited output vocabulary, especially if say you're running into a a word that you might never have seen before

and it's able to tackle a set of tasks that you know you wouldn't actually look at and think that um a neural network could actually handle at this stage. things like calculating convex hull of a set of points or the don triangulation and again the real challenge here is we don't have a vocabulary at all like you know it could be 0.1 is 42 uh for the x-axis yaxis is 12 19.5 but if we change that to 19.7 or so on it's not something we can really embed as a vector uh sly and we want to reproduce that exact point when it comes to the output so you can try this in a sequence to sequence fashion um you can just say uh output the ordering of the tokens that would give you the correct answer. So on the left 1421 is like the correct answer there. But you end up having a lot of issues. It has to learn about ordering. It has to learn about storing the correct numbers in its internal state. So again all these issues of information bottlenecks coming up again. Instead the pointer network can literally point straight to the correct answer or if it doesn't correctly point to the answer, the uh the loss actually tells it exactly where it should have been pointing

So it provides supervision directly to the model and we can actually see what kind of uh answers this ends up giving out. So the top left is the LSTM uh and it really does pretty terribly at uh you know convex hole. But if we can basically take that problem, make it worse and then throw it at the pointer network. It ends up doing incredibly well. Ends up getting the correct answer and only selecting the small set of points that are actually correctly on the convex hole boundary. Same sort of thing for the traveling salesman problem. it's actually able to do it pretty darn well. Um, and I mean that's, you know, MP complete problem

And in this case, it gets one little thing incorrect, which is the ordering of just one tiny triangle, which I probably would have gone wrong on my attempt in as well. This can actually be extended to uh your uh text pretty cleanly, pretty simply. And the idea is for each and every word, as we had attention before, we end up doing attention over um the input. And the word with the most attention basically ends up winning. Um if the word occurs multiple times such as in this case a bomber occurring twice, we can end up summing this mation and you know end up uh selecting the word depending on whether it's basically considered the most important thing in accordance to the question that we're asking. So pointer networks avoid storing the identity of the word to be reproduced at all. Um great way of kind of skipping ahead. And additionally, in most tasks, you know, the name is a placeholder

If we're asking who was the king of something, we don't actually need to worry about, you know, reproducing the name or the information that name contains. We're only interested in the name. Additionally, for these really long documents, we will never have enough working memory. It's, you know, if we end up looking at a book one day, we're just not going to be able to scale it that large. But we can easily do this kind of pointer method over it. And again it provides good attention good supervision as to where in the an in the original document your answer actually lies. So this can even be extended further to multiple attention passes as we were noting before. So dynamic memory network already does these multiple attention uh passes over its attention mechanism for complex reasoning and it's particularly useful for tasks where transitive reasoning is involved where you know the answer needs to be progressively refined

So this is work from Kiming Jong, Victor Jong and uh Richard Sosha uh from uh Salesforce research and the general idea is really quite simple, really quite cool but hugely powerful. Uh it's done over something called a Stanford question answering data set and that data set basically takes a number of Wikipedia articles and asks questions and the answer has to be a span of text. So it has to have a defined start and end point and a lot of these questions are actually pretty complex. um you know things like what does China's investment mean for Kenya over this you know large paragraph of text you're trying to find the one key salient point and the concept is almost like the dynamic memory network we have some document encoder we have a question encoder we have a co-attention encoder which basically if you think of it it's a pointer network with a start and an end point um that we keep updating and the specific implementation is a little more complex So I recommend you check out the paper for that. But it's it's really beautiful to kind of see in practice. So we have this start and end point and as it goes further and further along it becomes more and more certain as to where those start and end points should end up being. And in this case we're asking what did the Kenyon business people hope for when meeting with the Chinese? Uh and the answer is very close to the the ground truth answer. Um basically adding a single extra word at the start

So one really cool uh advantage from this is that the underlying data set squad actually seems to generalize pretty well to a lot of different tasks and additionally like the pointer network out of vocabulary um uh terms aren't actually a major problem. Uh so in the last like five minutes I'm going to show one extension to the pointer networks uh which is what if the correct answer isn't actually in our input uh then the pointing can never actually be correct. it can't point to something which has the correct answer. We could use an RNN but it has that you know limited vocabulary which yes we need to end up using but if the word isn't actually in that vocabulary you know we've got the answer wrong again. So one way of doing this is actually combine the two pieces of information. we have the pointer network and we have the um vocabulary from the the recurrent neural network and those are two pieces of information that we can end up basically ensembling and you can actually ask the attention mechanism to decide whether or not to um lean on one part of the model or the other in a kind of joint way. So in the top part we've got you know fed chair fed chair Janet Yellen raised rates and the start of the next sentence miss yellen is kind of the expected thing we're looking for. We end up performing attention over that original input as we've seen before

But we add one extra term here which is the sentinel on the far right. And the sentinel basically says, hey, I actually don't think the answer is in the pointer. You should give as much probability mass as I give you to the vocabulary. So it literally just decides to back off to the vocab softmax um depending on whether or not it's actually confident that the answer is in the pointer. Um, and this is important as the recurrent neural network hidden state has limited capacity and it usually can't actually recall what's in the pointer from a long time beforehand. So ensuring the pointer itself is what decides is kind of the correct way to go and it ends up degrading to a straightup mixture model. If the sentinel is one, it's only the vocabulary. If the sentinel is zero, it's just a pointer

Uh, there are some examples but I won't run through them as I have kind of limited time. But it really helps for rare words. So, you know, rare names or maybe uses of terms in different types of concepts. Um, Facebook also independently came up with a very similar tactic. They took a pre-trained RNN and then trained an attention mechanism over the top of it um as is and then enabled it to look back over 2,000 words of context. So, a huge chunk of time and they saw enormous jumps. And this basically indicates that even trained recurrent neural network models aren't able to properly utilize their history. um whether that's due to lack of capacity or issues with training

And finally, uh image captioning. So, taking one step even further along for image captioning tasks, we don't actually need to look at the image for most words. Um so, how can we indicate what parts of the image are relevant and when the model doesn't actually need to look at the image and does that actually mean the model can be um better at producing the sentence if it doesn't actually distract itself with the image? So this is uh work from a number of my colleagues um Ja Sanlu Kaming Jong Richard and also uh Periq from Georgia Institute of Technology and there are some amazing visualizations that enable you to see exactly what the model is looking at. So in this case uh a white bird perched on top of a red stop sign. You can see the bird is highlighted perfectly um and it's paying huge attention to the image itself. The sentinel gate is incredibly high. But when we get to the word top, it's almost paying no attention to the image at all. It's already pulled in that information

Um, red, it pays attention to the image again. And sign, it's again lightly paying attention, but you can see exactly where the attention is actually paid in these images and where it actually is useful. So in this case, a zebra attending standing next to a zebra in a dirt field. It's actually accurate on all those counts. Um, a young boy holding a kite on the beach. Again, accurate. And although slightly inaccurate, this is actually a woman. Um it correctly identifies a human um in the image and identifies a bike, the road, the body of water

So all of these things, we basically tell exactly where the model is looking. So back to our key idea, whenever you see a neural network architecture, ask yourself how hard is it to pass information from one end to the other? Are we actually pulling the correct information out? And uh is this the best way we can be utilizing it? The memory allows us to store far more than just the the standard hidden state size of H. And attention gives us a way of accessing this previous information and extracting the memory when it's relevant. Not only is it better for accuracy, it's also really cool for visualizations. Um, and they're useful debugging aids. It actually enables you to tell when the model is making the right uh answer for the right reasons. So, finally, uh interested in chatting, uh my colleagues and I are up at the booth um Monday through Wednesday. Um, and you can also say hi online at smear.com

Thanks. Thank you, Stephen. Um, we have time for a couple of questions. If you have questions, please uh raise your hand. I'll bring in the microphone. I have a couple questions. Um so you're talking about um looking at uh what the um important answers are given the question um by looking back using utilizing memory. Is there a way to flip that around so that the model can guess what the most pertinent questions would be to ask of what it sees? That's the first question

And the second Oh yeah. Uh absolutely. Um, so work from someone who hasn't yet been published. I believe it might be coming up in the recent publication uh, deadline. Um, it's a good question to ask because it can potentially be used to also bootstrap other models as well. Um, and so people are actually interested in playing around with it. It's kind of hard because uh, it's difficult to to work out whether or not the, you know, question it generates and the answer it's related to is actually correct. Um but it would also be a good way of kind of double checking again the knowledge that's coming out of this neural network

Okay. And then the other question is if uh you can utilize memory and uh context from previously that would generate I guess um post uh entropy of what's coming up and the entropy can be used to determine what's important to keep in working memory or or to toss. Um but if something early um is you know not useful that or changes you know that could change the entropy of the following uh concepts. Is there a universal quantifier to figure out what is you know doesn't change with respect to entropy throughout the entire chain something that you can always just say get rid of because we'll never need to use it or very rarely need to use it. Right. Yeah. Uh unfortunately that's that's a little more complex. Um, one thing you might have seen some visualizations of say LSTMs and their hidden state and how it changes over time

Um, those visualizations are incredibly cool, but they're also kind of painstakingly done. People actually analyzing individual neurons and seeing when they do and don't fire. They find that, you know, it might become more and more active as it gets towards the end of a line, which is kind of indicating, hey, you should, you know, add a new line here. Um, but again, painstakingly done, and it's hard to kind of, uh, produce kind of an automated way of analyzing these hidden states. when it does discard information, if it does discard information. Um, and that's kind of I guess one of the big issues with a lot of our different techniques. I think uh time for one more or Would you expect that attention would reveal anything interesting about semantic segmentation networks or is the output segmentation going to look exactly like the attention uh results? Uh so I will note I haven't done work on semantic segmentation. Um but I I don't think it's necessarily going to line up one to one

Um, a lot of the time the information like I mean hopefully it pulls in information to check whether or not it's actually relevant to the information at hand uh to the question at hand. Um, and that means that you know if it's doing its job correctly it's probably discarding some chunk of that as well which doesn't map to its you know conceived notion of what should or shouldn't be as part of the image. Um, but unfortunately I don't have previous background knowledge in that. Thanks. [Applause] [Music]