Why you can't afford to ignore deep lear...

data.bythebay.io: Jeremy Howard, Why you can't afford to ignore deep learning

Recording: data.bythebay.io: Jeremy Howard, Why you can't afford to ignore deep learning

Um I uh came up with this kind of slightly strong title because when I talk about deep learning uh you know it's so trendy right now that people assume that it must be full of you know, which is kind of how my mind works as well. And it's one of these weird things that's actually trendy, but actually even more important than people seem to think it is. It It's quite similar to the way I remember the internet was in the early 90s, and I was talking to everybody about the internet in the early 90s, and people were like, "Oh, you know, it's such a trendy thing. It's probably full of shit." You know, I was like, "No, I think actually the internet's going to be quite important." Um, I think deep learning will be more important than the internet. So, whatever you're doing, I I' I've focused this talk a little bit on life sciences, but uh uh you know, covers quite a few things. Whatever you're doing, I think it'll be important. I'm just going to keep an eye on the time. Um, so you can the nice thing about deep learning right now is that you can see it's doing amazing things

So, anybody that uses Inbox by Google on their Android phone has had the experience of opening an email up and having some pre-written replies pop up for you. And it's insane how good those pre-written replies are. And I was actually talking to the next speaker who's going to be talking about um UX and machine learning and so forth. Hunter and uh this is we were just talking before about this idea that technology this this technology this deep learning technology impacts the user experience and the user experience impacts the technology because this just wasn't possible before and now that it's possible you know if you have an email system that doesn't pre-write a reply for you then suddenly you're way behind the competition right the funny thing about this is that as little as six years ago The I will write your email for you was an April Fool's joke. Do you guys remember this autopilot? Um, it was an April Fool's joke. I think it was like 2010, 2011. Like the idea that Google could prewrite your email for you was so ridiculous that it was a joke. Um, and this is kind of what's been happening a bit with with the results of deep learning recently

So, another example, um, is Skype translator. So, you might have noticed a button has now appeared on Skype which will translate your voice in real time to six different languages and vice versa, which is kind of mind-blowing because again, this was uh science fiction. You know, do you guys remember the Babelish from the Hitchhiker's Guide to the Galaxy? And in that case, you know, I don't know if you remember this, but the babel fish was actually considered so ridiculously useful that it was the proof of the existence of God in Hitchhocker's Guide to the Galaxy, which therefore proved that God didn't exist because the proof of his existence means that he can't exist. But anyway, um so, you know, we've gone like beyond what science fiction writers thought was, you know, a reasonable potential uh outcome of technology. Uh it's we're even seeing it in in art. You know, this this piece here was generated uh automatically from the content on the left and the style on the right. Actually looks pretty certainly better than anything I could do. Um the interesting thing about these three examples is they've all appeared within the last 12 months

And beyond 12 months ago, unless you were very close in the deep learning field, I very much doubt you would have believed anybody who said computers will be generating artwork like this or doing real time translation of voice or writing replies to your emails for you. So clearly there's been some kind of extraordinary bump in technology applications in the last 12 months which we just can't ignore. Um so the technology bump you know as I'm sure you've gathered is is this algorithm called deep learning or this kind of group of algorithms called deep learning. If you want to know more about them there's a talk on ted.com by me um kind of with a wide introduction to them but I'll try and talk a bit more now. Um uh let's start by talking about where they came from. So they came from the more general area of machine learning. Uh so that was the invented by this guy Arthur Samuel 1954. Uh he wanted to make this IBM mainframe uh be able to beat him at checkers and obviously you can't write a series of steps to say here is how to be better at checkers than I am because if you could do that you would do it yourself

So he got this computer to play against itself thousands of times until by 1962 it could beat the Connecticut state champion. Um this was the first example of machine learning. So anytime you're not explicitly programming a computer with individual steps to take, but instead getting it to figure out how to solve something for you, you're doing machine learning. And there's obviously lots and lots of ways of doing machine learning. The interesting thing about um deep learning is that it creates the features automatically. Um so the um when you start with photos and run them through a deep learning algorithm, it will automatically generate layers of some semantic looking features like these edges and gradients and then you know pieces of the face and specific faces. Um this is an actual example of um some layers that have been selected from a deep learning network that was trained on faces. Um and nobody programmed it to say first create an edge and gradient finder and then create a part of face finder and then create a type of faceinder

That happened automatically. And so this is what is the key thing to know about deep learning is it removes the need for a human to program in features but instead it can literally learn them from nothing but the the pixels in this case. So that's a huge step. Um, obviously it means that it's much faster and easier to write these kinds of things, but also it means they're much more accurate. The most recent deep learning networks have over a thousand layers. So like over a thousand sets of uh kind of exponentially powerful sequences of of features just kind of like these three, but you know over a thousand of them. Um, how good are they? Um, they're, uh, as of a few months ago from this paper, they are better than people at recognizing the content of photos. Um, how good are they? They are better than people at understanding spoken language

Uh this was a result which maybe is less well known because it came from China and over here we don't notice China so much but BU is a company you definitely want to be familiar with. They're the kind of Chinese Google if you like and uh they have you know they're pretty much at the cutting edge of deep learning now. They're using deep learning, you know, for search, for advertising, for, you know, they they kind of were really ahead of the game with this. And now with speech recognition, they've got these extraordinary um particularly in Mandarin speech recognition systems. And they're trained what they call end to end with deep learning. And what that means is that uh their algorithms take as input a waveform literally that it takes just you know the the binary data of the the spoken um uh file and the output is what that person said and everything in the middle is automatically learned. Um, and at this point that approach from BU is the state-of-the-art in speech recognition and indeed better than native speakers, which is pretty cool. Um, in terms of what else can deep learning do right now, it can literally make you an artist

Um, this is an example of something called neural doodle where you draw on the screen, you know, your best attempt at uh uh some beautiful piece of artwork and it will automatically render it in whatever style you like. And so in this case, it's rendering rendering it in an impressionistic style. Um, you can see that this is allowing the computer to do something very different to what we think of computers as doing. You know, like you couldn't program something step by step like this. You know, it's really saying, I'm giving you a sense of what I want and I want you to interpret that sense. And it it does an amazingly good job of that as you can see. Uh so suddenly everybody can have their own Van Go. Um this is something I created online for free

Um so it's not even like these things are expensive or complicated. Um you can create drum beats um with deep learning. Um and my favorite one and so I'm just going to leave this running for a while while I chat is the create uh choreographic sequences with deep learning. Um I just this is so much more interesting than my talk. We should just watch this. Um so I you know um as Alexi said I I was at Kaggle until two or three years ago and so you know the nice thing about being at Kaggle is you get to hang out with the world's best data scientists and also the people with the world's hardest data problems kind of by definition. um because you know it's a company all about solving the hardest problems by finding the person who's best in the world at solving that problem. And it was through that that I realized that um increasingly deep learning was the thing which was solving all of these previously unsolved problems

And so that made me want to do something with this technology myself. And I noticed that, you know, the people that were solving these previously unsolved problems with deep learning were people that had no domain expertise themselves. So I thought in theory I should be able to do anything. You know, I should be able to pick a problem and solve it. And so I literally spent a year trying to find the most important and interesting data problem um there is. Um and after a year um I'm sorry we have to move on now. Bye-bye. Um after a year I came up with uh if this is doesn't want to move on

Here we go. Uh I came up with um medicine and specifically initially cancer. Um did you know in in lung cancer if you can um detect lung cancer early you have a 10x improvement in survival odds? 10x. So when people talk about beating cancer and lung cancer is by far the most, you know, deadly cancer in the world, we shouldn't be talking about genomics and, you know, amazing treatment methods or whatever. Just find it. You find it. Find it early enough, you have a 10x increase in probability of survival. And it's more than just survival, right? is like get rid of it before the patient even feels anything rather than having to put them through years of of suffering

So literally uh I started a a medical company and um and within a few months we had found uh you know basically applied pretty standard deep learning algorithms with a few tweaks to uh lung cancer data and come up with a state-of-the-art algorithm for early detection of lung cancer. um which was it was kind of shockingly surprising even though I knew in theory it ought to work because none of us knew anything about cancer or medicine on the team you know and so there's something kind of weird about um actually seeing this happen. So um this this turned out to be fantastically successful and um uh hopefully you know in fact it's kind of kicked off lots of computational medicine type companies that have appeared since then. So it's now, you know, just really what I most hoped to do would be to show people the power of datadriven medicine, as I call it. Um IBM's invested a billion dollars in this, you know, in the last um year and in buying a a medical imaging company. Um so and uh uh Deep Mind announced their first application area was in health and so forth. So it's really exciting to see kind of where that's led. Um, it's also interesting for me to kind of look back at my background

Um, which is maybe not what you would expect. I in theory am a philosophy major. I actually never went to a class. Um, but I do have a a major in philosophy because I did turn up to the exams at least. Um, and worked in um, business strategy consulting for quite a while and then I've been running startups for quite a while. Uh my most proud moment by far is getting to the top of the Kaggle leaderboard. um which um you know is my my most proud moment because that's like something you can't your way into you know what I mean like it's it's always easy when you have some success in life to kind of think oh I you know I got lucky or the vice versa if you have a failure in life to think oh people don't appreciate me you know so it's kind of nice to do well on something that's totally quantitative like that but I actually I mentioned this because you might have noticed something that's missing um which is my extraordinary medical expertise. Um, this, you know, this to me is what's interesting about really delving deep into data science and particularly into deep learning is that not just theoretically, but I think I've now shown empirically, somebody who knows really nothing about what they're doing can make some pretty significant strides through data analysis

And in fact, what I've learned over the last two years of working in medicine is not knowing what you're doing is in a lot of ways a benefit because you're um you don't know what's not possible, you know, and of course nowadays I spend lots and lots of times with time with doctors and radiologists and medical professors and you know, um I'm very deep in that world and I give talks at medical conferences and so forth. But definitely having that outsider perspective I think as well as the deep learning um tool has been critical in you know doing things that haven't been done before in this field and you know this field is a field that something needs to be done. There's more there's more than a 15x shortage of medical expertise in the developing world. uh the World Economic Forum thinks it'll take over 300 years to train enough people to meet the medical needs of the of the world. So, we absolutely need ways to diagnose and treat people more effectively and more productively. Um so interestingly you know since I um started doing this there's been plenty of examples that this is you know medicine's an example a great example of something that kind of any data analysis an analyst can get into and make a big difference in like so there was actually a Kaggle competition quite recently which was looking at um diagnosing uh the condition of the heart um for various reasons the size of the left ventricle is a particularly important thing to be able to measure very accurately. The winners of the Kaggle competition were some um hedge fund traders. And the um algorithm they came up with, you won't be surprised to hear, was a deep learning algorithm

And the algorithm is better than the best results shown by um humans, medical experts in the medical literature. To me, one of the really interesting things here and been really underststudied is what would it look like when uh these medical experts have access to these algorithms designed in a way to augment them as well as possible. You know, ra at the moment we kind of have these bake offs of kind of human versus machine. But what does it look like when it's human plus machine versus human? because we know um from many areas for example in in chess still a human plus a computer is better than the best computers at playing chess um you know even decades after a computer first beat um the grand the top grandmaster. So this is like one of these areas. There's so many areas in deep learning that are are shockingly understudied and underappreciated. And one of them is this big question about how can computers and humans work best together particularly given these extraordinary new capabilities of computers. And hopefully Andre will be talking a little bit about that straight after this

Um none of this is to say that um that traditional machine learning has not been useful of its own. It absolutely has. Um there there were before you know I came along and started doing this deep learning and medicine thing, there was plenty of computational medicine stuff going on. One of the most well-known and most successful being what's known as the computational pathologist or CPATH um by Andy Beck and Dafany Culler and others at Stanford. um and he's now at um Harvard. Um and they, you know, they showed some great outcomes. They showed a five-year survival predictive model of um breast cancer analysis. This is looking at the the slides, the biopsy slides

Um and the five-year survival predictive model from the computational pathologist was more accurate than the human pathologists in this case. Um so this was a great outcome. This is I think 2011. Um but it's very interesting to see how different the approach in this paper was. Um if you if you look at it they showed how they came up with hundreds and hundreds of features. So relationships between epithelial nuclear neighbors relationships between epithelial and strummal objects characteristics of strummal nuclei and stroal matrix. So I, you know, I've spoken to Deafany and Andy about this paper and basically they had lots of world-class pathologists and lots of world-class machine learning experts and lots of world-class programmers all squished together to kind of come up with what are all of the features that we can think of which we can code up which we can implement and that we can put into in this case a simple logistic regression um regularized logistic regression So if you look at the difference between this and what we did with lung cancer, you know, one of the differences is this is a whole bunch of worldclass experts from all of these different domains working for years versus a bunch of people who really have no idea what they're doing working for two months. Um, and a lot of the two months was frankly like just getting the data and stuff like that

I actually had a um guy come into our office a while ago who had worked on a sim similar problem. He was looking at digital pathology and he he basically presented to us the results of his PhD. He had spent five years and he showed how in this time he had come up with this algorithm to do a you know similar kind of um digital pathology computational pathology approach. Um and it's pretty impressive. if it didn't use deep learning because you know when he started his PhD there was no such thing such thing but nobody had nobody much had heard of it um outside a small number of labs and then uh on the last slide he said by the way last week I thought I'd try out deep learning um so I spent five hours replicating what I'd spent the last five years on and got a better result um so this is like and it didn't surprise me you know like this is kind Like because the five years of work was coming up with these kinds of things and programming them and debugging them and testing them. That's really hard, you know, and it's limited by your imagination. Like one of the cool things in this piece of work was um they discovered that the non-cancerous parts of the slide were very important at predicting survival rates. And this was something which um pathologists you know it's kind of just known by pathologists you don't look at those parts and it was only because you know somebody in this feature engineering kind of went oh what let's just throw it in there you know whereas deep learning has no such assumptions you know deep learning can find anything that's there and of course it can find it in much more nuanced ways like imagine a thousand layers of things built building on top of each other which is what we now have

So I'll give you a bit of an overview of other kinds of things deep learning is being used for. Um so um deep learning you know has a state-of-the-art approach to predicting infection rates. Um one of the interesting things is that the um the kind of network architectures that are used in deep learning now are becoming increasingly standardized. you know, so it's kind of getting to the point where you don't have to be a even a deep learning expert to kind of say, "Hey, you know, let's have a look at our data about infection rates in our hospital and see if we can predict what's causing them and make them lower in the future." It's still it's still got a ways to go, you know, but with things like TensorFlow from from uh Google um and similar kinds of libraries um or cafe uh you can kind of download something and get going. I wouldn't say quickly but but you know it's more of a you know small number of months u you know level. Um, another interesting area that kind of quite recently is drug discovery. Um, drug discovery was actually one of the areas that really put deep learning on the map because there was actually a calc competition sponsored by Merc for automatic drug discovery um, looking at which molecules would turn out to be toxic basically. And at that point there hadn't been any state-of-the-art results shown in deep learning

using kind of more structured data. Um, it had always, you know, the state-of-the-art results had really almost entirely been in computer vision. And Jeffrey Hinton and some of his lab entered this uh Kaggle competition um kind of at the last minute and killed it. They won it easily. Um and you know that was quite exciting because they showed you know something which I kind of suspected for a long time which is that deep learning is you know pretty is pretty much going to be able to handle any type of data almost better than you know traditional you know other types of machine learning techniques because of this ability to automatically generate features you know to compute arbitrary things. Um so uh over the next couple of years you know a bunch of folks at uh Stanford and Google kind of kept looking at this and eventually um this paper came out showing again state-of-the-art results in automatic drug discovery which is a huge industry of course. Um this is another example actually of a Kaggle competition. It's really interesting to look at what happens in Kaggle competitions because you see state-of-the-art result after state-of-the-art result after state-of-the-art result

So the um accuracy of finding the hick boson uh that was another Kaggle competition again one with deep learning. Um again you know the all all of these things are being done by people with no background in the area. You know these are not done by physicists. The previous one was not done by chemists. These are done by people who you know download a interesting looking data set and try their hand at it. Um it increasingly we're starting to see deep learning achieve state-of-the-art results in language as well. So anomaly detection in um newswire events. I mean you can see why hedge fund traders would be all over that kind of thing, right? So I suspect a lot of the best NLP happening with deep learning is probably happening in secret because it's such a rich source of of uh kind of hedge fund information

But we see some of it. One of the most extraordinary ones is um automatically figuring out the outcome of a arbitrary computer program. And this is where we can really see that uh deep learning can is an arbitrary computation device. Um so these computer programs were not run through Python. they were fed as input into a deep learning model and um the model made predictions. Um sometimes they're wrong, you know, so one digit wrong here, but it kind of gives you a sense of what's possible, right? Like where when a when a deep learning model can can run arbitrary code. Um, another example of this kind of thing actually came from some of the earlier work from the deep mind folks which sure a lot of you have seen this. They had a deep learning based system play computer games and they played computer games by having access to nothing but what color is each pixel on the screen

They weren't told the rules of the game or how to control the game or anything. Um, these were for uh many Atari games. Um all of the ones above the line are ones where that system was able to beat the world's best human player. Um so it's interesting this is not you know before deep learning any one of these would have been 10 years of work to you know build by hand something that can play that game automatically and it wouldn't have been as good as the best humans. you know, now we have one algorithm that can play all of these games and indeed went on to play go and a lot of you, I'm sure probably all of you would be familiar with this result. A lot of people don't quite understand how this result happened. So, it's interesting to talk about how deep learning was used here because some people didn't necessarily see this coming. Um, the reason that Go is challenging to play with a computer, it's not just because there are so many possible moves, but let's say you could study eight moves out, right? So, you try every possible combination or whatever of eight moves out or nowadays people use these um Monte Carlo based trees um to kind of do that in a more thoughtful way

But you've got a bunch of, you know, millions of possible board outcomes, eight moves out. Which one's the best? And this is the thing which nobody really knew how to do. So you would have, you know, millions of possible boards and the computer would have to decide which one it was going to aim towards. But it's very hard to score. You know, it's very hard to say and go, is this board better than that board? In any kind of rules-based way. A human can literally look at it and say, "Oh yeah, this one has more energy or this one, you know, is too weighted towards the top or whatever." You know, like they could they would all agree on the sense that they get. So what um the deep mind system did was that they put these board configurations into a convolutional neural network. The same kind of deep learning network that is used for computer vision

the one I showed you earlier that was the state-of-the-art for computer vision. They used basically exactly the same architecture and they told it you know these boards ended up winning and these boards ended up losing and it learned it learned what a good board looks like. So the deep mind system now does something a lot like the world's best go players which is it can look at an arbitrary board configuration and say that looks good or that doesn't look good. And we don't really know how it's doing that. You know there's a billion weights in there. Um just as we don't know what's going on in a human's head. We just know that it's learn to recognize these patterns. So that's basically how this um this system works

So it's kind of an interesting example of how you can get perhaps unexpected results like when people talk about computer vision the idea that that exact algorithm could go on to beat the world champion at go you know perhaps is a little surprising but when you when you hear it it kind of makes sense. So as you guys probably heard um this week um this is all about to take off even faster. Uh Google just announced much to everybody's surprise that for the last year they have uh actually been using a piece of custom deep learning hardware um which is 10 times faster than anything that you or I have access to. So um up until this um everybody who had been doing anything useful with deep learning had been doing it on GPUs and pretty much always specifically on Nvidia GPUs because with their CUDA programming model it's much easier. [Music] Um, we've kind of known in the community for a while that at some point custom deep learning hardware will take over because the fact that we're using GPUs for deep learning is a total accident. You know, they just so happen that the kind of things necessary to play a Batman game are similar to the kinds of things necessary to um back propagate through a deep learning network. If you designed something from scratch for deep learning, it would be much better. Um, I did not know that actually it's already been done

There's a company called a startup called Nirvana that's been working on custom deep learning hardware for a few years as well. Um, there's uh also companies that are doing work on um approximate computing. So these are are chips which like when you add two two and two together it'll give you fourish which turns out to be 10,000 times faster than something that can do it exactly and actually for deep learning it turns out to be better like a bit of noise in calculations actually gives you better results. So the the stuff I've kind of shown you is so much the tip of the iceberg. Like the more I've got into deep learning, the more I've realized we have no idea what we're doing. You know, like it's uh for 20 or 30 years there was only five labs in the world even working on it. You know, they weren't getting published in any conferences or anything because the whole machine learning world was very anti- neural networks. um since 2012, you know, with the image net results

Um that's gradually or actually probably quite quickly all changed and now it's hard to get into a conference if you're not talking about deep learning. Um but if you think about it, that's only enough time that the people in 2012 who were just starting their PhDs will be starting to kind of come out of their graduate programs now. So the amount of people with deep learning you know kind of uh graduate level deep learning capabilities will be really spiking over the coming years and ditto with hardware you know the num you know a small number of companies have been working on deep learning hardware for a couple of years um uh I think particularly with stuff like this showing what's possible that's going to take off. So everything I've shown you in terms of the deep learning uh kind of momentum at the moment uh you should think of I think as being similar to the internet momentum in 1994 you know it it feels like a lot but it's it's really not you know it's it's all still to come. So I got five minutes if uh anybody wants to ask any questions. Okay. [Music] Thanks. Uh in fact in yesterday's um evening panel uh I raised this question to the panelist about uh are we dealing with uh deep learning as a toy today but it can quickly turn into a monster and as we speak and learn from many deep learning experts where they think that we are reaching a tipping point that we really don't know where all this is going

what kind of threats or challenges you see at large for humanity? Well, the the biggest threat to me is um income inequality. So, um, the, um, and this is something that I talk about a bit more in my talk on TED.com, but, basically, the industrial revolution led to 70 years of of a reduction in median income, which I mean, now we're all glad the industrial revolution happened, but it was a lot of upheaval. Um, you know, I think that happened because machines replaced humans in terms of providing energy inputs into processes. As machines replace humans at providing intellectual inputs into processes, you know, I think that's going to make a much bigger difference than the industrial revolution. And it won't there won't be a leveling off of that technology. It'll just keep growing. So um they'll you know it's going to naturally result in a few people having kind of all of the economic value ad in their hands. So uh in countries like the US or particularly the US where you know we are so into you know labor scarcity and incentives to work and stuff like that you know we could end up with huge social problems

When I say huge social problems, I mean as in, you know, people with guns shooting each other level social problems because there won't be enough work to go around and it's going to happen more quickly than anybody expects. You know, even deep learning experts have been surprised by the progress in the last 12 months. Sort of a sorry, sort of a medical specific question. Um given concerns around malpractice and things like that like how do you deal with like setting aside the technology side more the human side like how do you deal with getting doctors for instance comfortable with the idea that you know something's going to be run through sort of a deep learning algorithm and it's going to spit out a result and even though you can show objectively that's more accurate than what a human would do getting people to trust that you know so I mean that's something I've spent a lot of the last two years working on and thinking about um and of course this is all about being effective at augmenting humans not replacing them. So if you give them access to say radiologists access to a system that highlights in red every medically relevant abnormality so that they can like not miss them and see them more quickly. At first, yeah, they might, you know, totally ignore them, but after a few weeks, they'll notice that, okay, every time all of the red things were labeled and a couple of times they had missed something and then they noticed this red thing and they're like, oh god, I'm glad I didn't miss it. And, you know, they'll start to generate trust in it. And then the you know also the um uh clinical journal articles will you know which are really very important in in medicine will show the kind of AB results of doctor plus computer versus doctor and um and then I think what will happen over time is the insurance companies will start to say you know well people using this are twice as fast and twice as accurate so we don't want to waste our money on people that aren't using this approach and so it'll you know it'll gradually Naturally the trust will come and then after the trust will be the forcing function I suspect and you know at some point the NCCN or whatever will start to modify the practice guidelines to say you know well all of the science shows that this is a much going to increase people's outcomes at a lower cost

So you know eventually it'll get into the practice guidelines I expect. Quick question for you about um the tools that you may be using. Um to develop some of your deep learning algorithms um you know I've come across things like deep learning 4j [Music] um can you tell us a little bit about what what you're using in in your startup and I mean I I'll preface this by saying the state of tooling in deep learning is terrible. you know, there are no there are no tools I recommend. Um, there are tools that I am less unhappy with than others. Um, I've done most of my work with Theano. Theo, for those of you that don't know, is not specifically a deep learning framework. It's a general framework for um representing computation expressions and then um compiling them for the GPU

It's in Python. Um and then people have used that to create you know pre-existing computation expressions representing various neural network layers and you know so forth. So is great for like the stuff I like to do which is really digging in and trying different things. Um TensorFlow is similar. Um I'm also very u fond of something called MXNet which is a similar kind of thing that comes out of a largely Chinese group. Um, yeah, I'm not but I'm not very happy with any of them, which is why I'm kind of moving into deep learning research now because I want to build tools which are much easier to use and much more expressive and much more pragmatic. Um, you know, there are things that everybody in the deep learning knows that you always ought to do like for example data augmentation. um you know it's one of the most important things in deep learning is to take the incoming data and uh kind of randomly change it in various ways like if it's an image kind of rotate it and flip it and so forth and you know to me it's crazy that there are still no systems that you know automatically handle that for you and you know still most papers I read from non-deep learning experts that use deep learning don't use any data augmentation because they didn't know about it and so you know the current state of tools needs a huge amount of work to say the least

Okay, so I think that's about my time. I'll just mention there are I did notice a couple of talks later today that look at um deep learning in medicine. Um one of them interestingly is actually in the same field that I showed earlier which is um looking at lung cancer diagnosis. Um, we when we did it, we made it particularly as hard as on ourselves as possible, which is there's kind of two ways of doing a a CT scan. You can either inject with contrast, which kind of makes, you know, highlights the the vessels and stuff, or not. Um, and it's actually a radioactive dye that they use. Um, so we actually took images that were done without contrast as our training set and then used images that were with contrast as our test set. Um, which kind of is a great example of how powerful deep learning is

You can kind of train on a totally different kind of image. Um, so anyway, it should be cool to see hopefully some more detail about uh another group who have done worked on that. And then somebody who I do know a bit more about what they've been working on is uh the cardiogram guys will be talking about some um recent research they've been doing on uh predicting uh heart failure events from um uh I can't remember it might be I can't remember it might be ECG data or something like that but anyway so again with with deep learning so there's a look out for those couple of uh should be a couple of interesting talks about deep learning applications in medicine later today as well. So thank you. [Applause]