sftext.org: Alexy Khrabrov interviews Gregor Stewart

Recording: sftext.org: Alexy Khrabrov interviews Gregor Stewart

hello everybody I'm Alexa crab Roe for the organizers of SF text and Hugh are on location at melt water which is a very interesting company so today we'll have two talks on the back from meltwater a month Gregor from bases technology so Gregory's VP of Product Management I vices technology they work with not water and just do a few informal questions about Gregor he's interested in in NLP and basic knowledge welcome to a subtext so so you mentioned before when we talked about your academic history at any more American kind of outline a little bit to us how we got interested in NLP what do you do before and how you came to basis and what makes it interesting for you so as an undergraduate I did philosophy and spent a lot of time doing philosophy of language and it did that at a master's level and I started a PhD in philosophy of language and I thought this is not quite right so many years afterwards you know having had an interest all the way through childhood and computer science and so on I decide to go back and get a masters in NLP from Edinburgh mhm and essentially the connection was there the very sort of high-end theoretical understanding of theories of meaning and things like this mhm I wanted to turn that into a very practical you know set of things that I could do no me no understanding language in the real world let's say building systems that could understand language that was the motivation okay it's of interest because you know when computers first appeared there are no computer scientist so the people got together they were physicists mathematicians philosophers and linguists writers kind of so so you're kind of cover into at least of these categories okay so which is Ernest also so Universal van tomorrow is a very strong school in enamel p and artificial intelligence so what makes it such a kind of hotbed of innovation kind of so many great people is there anything special about the way they teach or the like the way they got together the labs like what makes it such a interesting place for LP oh so the fact that they have in the school of informatics which they made from the conglomeration of many different departments like linguistics and so on hmm and the cool location of all those people in a single building at least in the time i was there you know it's a that's sort of stoking the fire of if not innovation at the very least collaboration right which i think is the lifeblood of things like machine learning and so on problems problems that are important unity techniques and you need innovations on both sides right being the new problems bring me new techniques or anything I think that was you know kind of ideal fusion as I may be mentioned you before it took him a few years i think for it to to Kindle properly um but I I think that it's improving itself so I think people are attracted to Edinburgh in particular because such a beautiful city that's number one yes it is and the quality the faculty they've been able to attract for whatever reason I think you know what one of them being that is such a beautiful place to live mhm mhm is is a standing and so I'm very very happy to have spent time there right cool yeah I have a long-standing plan to do a meetup or text conference in Edinburgh so now it's only increasing so so basically it then so you move between academia industry a few times right and you ended up in industry and I can follow a similar path and kind of i'm wondering what you know how kind of did you kind of end up in in the industry and what do you want to accomplish being in a kind of industrial company serving customers right so i think that academic life offers you great many satisfactions one of them is that you can spend a very long time pursuing a particular problem and you're not really constrained in you know I guess how much time you spend on it or really I mean other than funding let's say but you can really devote most of your time to that I think in an industrial life you know in a commercial setting you do have lots of other things especially an executive level so for example the amount of actual product development I might get to do right in a year is is quite small compared to the amount of application to a particular paper I might be writing in academia so I think however one of the difficulties is that very useful things in academic life would be considered cute I remember you should we'd laugh what is it to join him or one of my teachers said to me I came to of an idea for doing something about pdfs and i know that you you know you work in that area and i said we'd be amazing if you could do the following things and she kind of looked to me and said that's cute I'm going to get you're not going to get any papers out of that and so there's there's there's a very there's a there's a strong tension between doing things that are cutting-edge you know and and doing things that are worthy of papers and the two things are not always the same and i think in industry at least there's the reward monetary let's say but also the satisfaction of having millions of people perhaps use your you're right comes your products right um that's not really something that you necessarily get from from academia very few people will ever read your thesis yes yes that's right yeah i'm kind of you know I'm you know I'm receiving the research get you know counts of my papers and that yeah that doesn't stop yes so yeah we reach a hundred something like they give you a big award or something like this I know so but but you know I should I'm very interest in this and I you know I think it'll p is a very hard area because you remember right the whole day I promise was about language and the thought computer second understand language and then there was this I winter because we realized this is really far often now I kind of back right on the cusp of you know personal assistance and and but it's still as predictions with no right this is still far far away from intelligence and so what was this really interesting to me in this meetup so this meetup is kind of started people doing something with products right so we end up on data products is no data science the science is powering but the product should be bought and so it's a very easy to miss much harder problem then then actually kind of advanced in the frontier of science because you need to make something works in the real life so in a way like again some knowledge of the real world so and i know the basis is an established company and it has government customers it runs and so on DC conference and kind of in a so i'm really curious what you take what really works so some stuff we know in theory with learning school we're different kind of grammars it right and you can in a little Chomsky right the question is you know we cannot implement it so by so basis you guys are implementing something which you want to sell and they want to convince people that it works so from kind of your standpoint what really works what kind of NLP can be really commercialized what kind of stuff people really pay for because it really works for them it kind of improves their own life it improves their own business what do you see really working so of course the foundational things that we've built you know which essentially make use of linguistic concepts you know which allow people to manipulate the text that they have at least to try to do useful things mhm these things are in some ways relatively straightforward well understood and the things that that make them work are you know mechanisms like finite state transducer and so on and these are very robust technologies and things like classifiers um you know sequence prediction these kinds of things these have been incredibly powerful right as building blocks labeling of all sorts right and has been very useful the vectorization of of words of phrases you know of larger text etc is also proving very powerful um however the kind of the larger promise of that I think is is yet to be proven the difficult part about it is although it provides a rich set of information in a very dense form you know the it has a difficulty in that the developer write often doesn't know how to use it in the same way that they might say I'm looking for nouns sure I a noun right so show me vectors with the following properties or whatever and very few developers actually have what is it either machine learning knowledge or optimization knowledge and so when it comes to building systems some of these more esoteric representations which are actually more powerful and the systems which produce them are less useful developer to developers and what we see most people building systems with av's very robust components say take linkedin or someone else you know entity extraction to very it's a very well understood problem for example and then still something people buy because actually does something useful we were just talking about this earlier mm-hmm um you know when you think about something like pronoun resolution you want to take you know I mentioned of Barack Obama when someone says later he has no one to relate that back to Barack Obama is most of the time we just do it by proximity right what was the most recently mentioned thing you know when you maybe have some skip you know in idea in there you know it skipped one or something numb or it has to match in terms of gender and that's about all you can do if you want to do it very speedily or you can parse the entire thing you know traverse the dependencies etc etc and of course that's that's much less robust right yes um but when you say to people you know why do you care by pronouns you'll say well because I care about something else and they'll talk about some task no level thing they have and just to take it back to the the vectorization stuff you know a deep learning person you know I would say or a representation learning person would say what's a pronoun mm-hmm who knows what that is right yes we just got these patterns in the data and if you see these regularities and it helps you do this job you can do it that's very difficult for some folks to grasp and I don't think I think it takes time to to to bring into your practice right however I do see that I do see that happening usually so can you do pro resolution in industry well you can do it to a commercially useful and degree right which is to say that and what's marvelous about all this thing is all these things is that most people right even when they're writing in these very odd genres like tweets and they're right in a way which is designed to be understood by other people there is some greicy in a maximum right which is hopeful try to be understood right um and and so they do things like order mentions they mentioned someone and then they use a pronoun and so on and so forth so only tend to mention a pronoun and they match the genders and all sorts of things right so they do these things in a regular way so an awful lot of you know that stuff can be recovered by these simple techniques because in some ways you must think that's what must be going on anyway yes right you know and so you should do customers and the busy are satisfied with kind of most of the results most of the time mm-hmm and largely I think that what is difficult isn't when they want to do something that's well beyond the vines of what those a grits will let them or the accuracy of those you know devices will let them do so you say for example you want to recover all of the you know implicatures of a document right you know urine after parcel the text you know put out into some normalized form and then do a whole bunch of inference and something of course this is a you know what is it a non non halting problem right I just gonna keep doing it for almost ever right and so you're often having to school people and say what's your real desire here right you don't really want an nth-order representation of this text the right what you want is to achieve this end right and a lot of the time that can be achieved by something as simple as giving an example people were saying while ago people are asking us for can you find forward-looking statements in it you know in in in financials right and oh yes I mean they're people can understand them people can recognize them you know we can we can build a model like it so how are you going to do that are you going to parse it and everything I don't know let's try a bag of words you know in a classifier right yeah and it says future yes ever say maybe yes so these kinds of regularities and and wonderful thing about language that people do try to be understandable to other people is what allows us to do you know a good job even though a lot of the time the technology is fairly rudimentary interesting oh this is kind of reminds me of undress plug ins kino that rich is a silent issue both attended yesterday and you know he had this for rule of data the rule number one was start of the question not of the data right it's almost kind of you know confirms what they're saying because instead of thinking what is the ideal full representation of this text the question would be like what is the question you want to answer by looking at this text yeah right so she was very cool interesting so so okay so in terms of parsing right so I mean we kind of tried in all kinds of open source parsers right and that they extremely slow right so basically in order the seconds I mean on the conventional hardware so do you think in a specialty web scale I mean it's really feasible to parse everything or do you really have to get by with certain heuristics estrela parsing or just tagging you know do you know that no like other tasks you know web-scale kind of data side so people really have to parse everything I mean so I think that parsing is coming along in in great strides in terms of its efficiency is computational efficiency I think in the same way that we recognize that FST could deliver a great deal of the recover a great deal of the interesting structure you know even though they're you know of a lower order than what we consider language to be on the top ski hierarchy let's say um was it but fully parsing things you know has has come along greatly in terms of is algorithmic efficiency and also the hardware that you cannot implement it on so you might look at something like puck I think num which is a way of compiling you know what is it now it's time for grammars at some other grammars on two GPUs right and that parallelization right along with the original efficiency i think is taking us to where was seconds per document you know it's my you know 100 milliseconds per diamond right um now you the quality of the parse may be slightly lower but as we said what is it if you're if you only consider it to be an information source not the one true representation of the document hmm what is it and you will see so sure and a bunch of other people using these parses as pieces of information refuse that they use or to learn a representation which contains parts of the parts right say do things like sentiment analysis all right and this is it's proving to work quite well so well that's showing is that only certain parts of the prayers are important right some of the time right man and so but if you can get it very quickly you can put it in these other methods and make use of the parts that are actually accurate interesting so just let's generate a whole bunch of features right and kind of let deploring figure out which is which 10 rounds are you so let's go in any contingent things no this is really cool so so in your customer directions what kind of things do you see more like what do you need to explain to be most of the time or the kind of most interesting things to share can if you see recurring in in customer direction Oh interesting um so often explaining the limitations right um you know saying what is roughly possible right I think people read enough lot of wired and MIT review and and get wonderful ideas also AM you know um people often ask me what does Watson do right I I say what's not a person so right I can't tell you but so that soul imitative you know I comes right let's say what works what's possible the other thing is just all so what can be done right tell me give me use cases customers are very often feel like they should use you know this technology or that it can do something for them um and offer and the question is what do you think it can do for me right I'm is there's a question it's moose often asked they often have a problem a specific problem that they talk to you about like they say that sometimes motivated by an over overblown understanding or an overblown expectation right well you can achieve let me actually get right down to it there they're kind of excited about the things that you can do take mel water for example we were talking about really linking it's not a cutting-edge task or is it it's it's really well understood um you know it's state of the art is not great yeah but it's still I'm pretty useful and super bored this is like the mainstay of the business yeah right so so it's a good match yeah so um you know taking entities taking entity mentions i licking them to some knowledge source and then using the structure of the knowledge source and the informations already contained in there to do something more right with the information that you have in there already right yeah interesting so these are these are things that customers often asked is about but the last thing is that how do we integrate it right what is it how does it work at runtime right you know a drama performance right right how much can I get through it that's right and often you know a corollary to that question is often does it work on tweeks right right and so people often pick the hardest text type to operate with first and then we look at the text mix and typically this is only a small part of it right right not with obviously met water it's a large part right right right interesting so so it's a decision so in the way when you explain technology how often does it change the original problem like so so it looks like you know the final setup is is a kind of collaboration right between customers needs and their capabilities as you explain them so how often do you see would you like say you usually stay and kind of satisfy the regional requirements or most of the time they actually change as a result of this interaction it's interesting yes they do often change and they usually change in a way which makes it easier for the customer to implement right you know what is it easier for us to sell them something obviously that's natural is we're just adapting to 10 we're in training right but also I think ultimately it generates new ideas right you know so we solve that original problem and then they start to see the other things they can do right now with with the output that they generated there and one the fascinating things is just how much business value you can generate from very very simple technology I'm sure your waiter it's like features in your own products right where you think god damn or people is this the thing people are using right you know and they'll say because actually as it turns out this is one of the things that you often as a company that does special specializes in NLP not every problem is a language problem mm-hmm not every problem needs to be solved I know every problem needs to be solved with language technology right and so many customers are come to as you know expecting to need you know entity extraction and various other things you know all the way up to relationship extraction and it turns out they don't that they can actually do it using structured sources mmm or or a little bit of computation interesting is it so so you do a lot of different people or I don't have like linkedin and pinterest right have some really cool customer so how do you integrate the feedback like what what parts of your products and business changes as a result of getting more customers been exposed to more problems Micawber amo texts great question so we have a large number of I guess low level requests right so I would say you know you build or essentially shallow partners and they may snakes until we get lots of quest questions about can I have you know Dutch pronouns come out a different way this lemma is wrong etcetera cetera and those things you know just comment on a constant stream and and they get divvied up you know among releases you know this is standard ordering track which is to say you know how important is it for the most important customers um you know how many customers are interested in this and so do i think this is actually just a genuinely useful fixed like for example the FST was over producing it was producing lemons like analyzed was coming as analyzed ice this was a long-standing bug what you'd seen in JIRA three times and I'm thinking no we should just fix it right it's to rule somewhere in arrestee right so this so those are those are the kinds of things that come in a constant stream and then I have what you might call major enhancement requests right which are sometimes hard to differentiate from true true true product requests right you know me any product a lot of the time it's problem oriented I'm trying to do X with you know this product and it's not working right you should fix it so for example a customer might say you know I need a Greek lish Greek lish is a Roman eyes way of writing Greek right much of the same era bz is a room a nice wave right in arabic ok and they'll say I need a Greek lish entity extractor and you'll say mm the market for a Greek leash entity extractors tameka tiny are you saying that the greek entity extractor doesn't do a good job for you and and so what they really need is just transliteration right right you know which would at least get them a story it's not translation right I'm by it's probably good enough get ridiculous back to Greek yes and then and then do it and it will do a decent job so sometimes we get these requests for new products which are really enhancement requests for other things like you know I want I want you do you agree Greek to you know Greek lished a Greek transliteration right in a decent way but the most exciting ones I think are the problems we simply can't solve with anything in the stack which you know force the rd team and the rest of the product teams to rethink what is it say some of the output that they're they're generating or the techniques that there nilin themselves so one of the exciting aspects of the work I do is directing the R&D team to towards you know some of these more you know esoteric techniques that now doing that in the future when some of these problems that we can't touch night you know when these things become robust will actually be able to roll loose forward so cool well that sounds great so and I'll guess like you know couple of questions you know one will be summation Watson right as what's this kind of its it's interesting right so it's kind of it's now industry has a face right and so everybody knows about it IBM was doing huge work in marketing kind of so would you so when they ask you you know what does Watson do so do you think first of all is it helpful to the whole industry and how everybody else should kind of you know compete with Watson or how like what's was our collective take on Watson is non Watson yeah so personification i think is a very powerful technique in marketing right so making the person feel like you know this body of technology which is what it is really right and people obviously us is doing something for them and has a certain character right and i think the character that they've developed with Watson is amazing it's very you know it pervades the same we say Siri right instantly understand what Siri can do for them n its intent yes what's it is supposed to help you right that's they call it what's in the homes right know what's in helps that's right so homes is just a supercilious a solo that's right so so the the the the parrot that power the power of that personification I think too in general people a desire right at oak could what's in help me with this yes can Watson help me with that race so very useful and I think it's done the industry a great service ring I had a nice chat with the manager may know some of the Watson cloud stuff and you know they base many of the same problems we do you know people ask for functions they don't have they need improvements etc so in some ways it's very similar and I wish that we had thought of you first right right all right well you can look on Watson's British right so you can come like like American version yes of awesome yeah we're sort of helpful on this post or if I score exactly do i do think it's I do think that's the way to get people to understand what is it the power of these technologies and I know that other people and you know work hard I mean Cortana such an odd rain right I'm you know it doesn't it doesn't speak to one in the same way that the serie does less right that's right that's right yes something from Mars yes something hey I don't think kirtana can help me I think he's a drink or something that's right yeah it's kind of interesting so okay no that's I found it personally you know Watson is very intriguing and kind of and it's definitely they have this very nice videos and you know obviously it's just kind of the making inroads but it's very interesting how they kind of make people think that you know this is a I effective whether this can solve all the problems and then the structure so they're kind of the focus on different businesses so the problem have the main expertise kind of you know learn for specific verticals right so I think the result of interesting notions ah so I think I'll wrap up with kind of model you sure so you guys are working lots of cool customers you see a lot of problems this customers in turn are advanced in 20 years of industry right like pinterest and linkedin obviously they are web-scale the few months of data all kind of stuff so what do you think in for you guys what's the most interesting right like when you're pointing to the future where do you think we're going to be what a preparing for what they want to to to kind of focus your energy on in order to be there when it happens so I think the dialogue or communication with people in networks right I'm still untapped right to the patterns in the way that people talk with one another who they talk to who they contribute whose projects that contribute to and so on and so forth so I think Marie Wallace at IBM talks about systems of engagement and you know and one of the most exciting ideas you know in that whole set was having technology that watches what people do that watches who uses their stuff what they write it cetera you know in a in a passive and non-invasive way I'm talking about their business you know materials and so on and then provide some feedback I think one of the most powerful things that you know you machines can do for people is to give them a notion of how to improve right so tell people how are you you know how are you deficient or what are you doing well right um you know are you and this is wonderful dashboards that they had you know which indicated you know you're really great at people really love your work I'm or is it but you don't talk to many people right right right and to give you some sense of what it is to be a good actor right um and to bring a metric to that and I think that whether it's watching where people move around you know like human eyes and a bunch of other companies that do that that work to track people in the workplace or to see what you do to help you it's more like the conversation that Monica riccati started yesterday rich data summit not about helping people help themselves yes and this is something which there's an infinite market for as you know you know self-improvement is something which you know people are what being willing to pay for since the time the Bible is written especially this yes exactly so and it's also speaks to it because there are groups of people who aren't just you know people wealthy people trying to get more wealthy but people who genuinely do need to improve in one way or another right whether it's losing weight or it's what is it you know having a better relationship with your children or whatever it is these are all important things and I think they're there are roles here for technology to play and so the underlying technologies which allow other people to understand themselves people to understand themselves and other people better is where we you know tend to place ourselves ultimately that means understanding the information that's in text better mhm but I think that's the the farsighted go this is super exciting I mean this is very interesting I think you know this is this kind of this is where you know technology can shine this is the best application so i hope you guys build sooner and don't talk about it right so always welcome to come back and talk at the subtext Thank you Thank You Roger thank you some