ai.bythebay.io: Bots Panel

[Music] uh I'm Pete scarok co-founder and CEO of skip flag uh we're a startup working in the Enterprise space and basically we turn conversations into knowledge uh you can go to skip flag.com and sign up your slack team Andy out uh we're still waight list but uh coming out soon um so this panel is going to be about bots in AI uh kind of that intersection uh it's a hot area right now um and maybe by the end of it we'll figure out uh the answer to Bradford cross's question of whether Bots are going to go bust in 2017 um with that I'll let my uh panelists introduce themselves oh you know yeah hi I'm David Hall I'm a senior research scientist at semantic machines uh we in Berkeley and Boston uh I co-founded it uh together with my PhD advisor Dan Klein and a number of other really excellent people and we're building a Next Generation uh conversational interface my name is is carel I work at a company that is called spoken H spoken is a cloud infrastructure company that provides infrastructure for customer service and as a result my wall in the company is analyzing all of this conversation that happening on on the infrastructure so we are building very very large system that can analyze millions of hours of conversation on a daily basis hi I'm Andrew berer I'm the CEO of aumo company here in in San Francisco and uh we help uh businesses create conversational interfaces to to their products so primarily helping companies create chat Bots and voice Bots connected to their existing product hi everyone um I'm Dennis Yang um I'm one of the co-founders and chief product officer for dashbot um this is my 20th year working in technology in San Francisco um but and what we do is we provide conversational uh analytics or analytics for conversational interfaces more commonly known as chat Bots um and we most recently push over uh I guess processing 1.2 billion messages hi everyone I'm sonil Gupta I did my PhD at stranford with Chris Manning graduated in 2015 since then I've been a researcher at Viv Labs at Viv we are building an intelligent virtual assistant so think about Apple Ci or Kotana so a user facing assistant and more importantly we're also building a platform for thirdparty developers to teach Vi about stuff they want to get done at w that's great so that leads uh into the first set of questions I wanted to talk about which um are about Bots and Lear so um how far we let me put this first one to David let's start over here um how far are we from Bots that understand arbitrary natural language questions and can have a multi-turn dialogue with people and can you say a little bit about what what a multi turn dialogue is sure um so a multi-turn dialogue is a um a sort of sort of what it sounds like where you talk to a computer over a sequence of multiple utterances and sort of collaborate with the computer to figure out what it is that you would uh what you wouldd like to get done um there's a lot of follow-up questions um things like that um and in terms of where I think things are going to go um I think that we're going to see a lot better support for multi turn dialogues um in the coming year or two I think that there's going to be a very long time before we have strong AI or sort of strong natural language interfaces that understand absolutely everything you're going to say especially dealing with common sense in the real world and stuff like that but that you will be able to actually sort of get something done something done with a computer um without having to sort of carefully phrase a command um like you do today uh is what what's your take how far are we from multi- churn uh dialogue Bots no I agree with David I think that you know most of the most of the know the things that we we are seeing today these are really cool and nice stuff but in terms of b or some kind of a conversational AI system that can truly understand you know a human language create some kind of you know Multistate dialogue system we are really far far away it will take some time there is some kind of a technology limitation with all the advantages in deep learning for speech and natural language processing there is a lot of there's been a lot of progress but there is still a lot of work needs to be done so now what's your take uh after working with Chris Manning on uh uh dialogue and speech what what's your take on on getting this to be practical so dialogue is a really really hard problem as we all know and I feel we'll have some kind of system that would be useful in some ways within a year or two um now considering what's your expectation level so if you want it to be like a human that understand all your desires and constraints in whatever format you want to say it in that's going to be a few years I think it would be at least 5 to 10 years what about like interrogating there are some interesting papers recently in the world of images is where uh was it maluba I think I don't know if people saw that um startup was acquired by Microsoft um I I think they were doing things where they would train they'd have a a a set of humans ask questions about a picture and then another set of humans answer those questions to train uh basically an oracle about a picture that then the AI could answer questions uh opped to it like do do you think like in the near term we'll be able to like have a document or an image and Viv or something would be able to answer be interrogated about it till we have the training data I think generalization is still a big problem so till we have some kind of training it's in the similar domain it can be spoken in different ways but it has to know about it it has to model the use case I think we would be able to answer those questions but if we just assume that the system would have some kind of Common Sense knowledge that it should just know understand things without even training trained about it then I think we are pretty far away from it so so when you talk about training data Andrew um is that is that really learning or is that just memorization so if we give a lot of examples of responses response pairs is is that really learning or is it is it just like building a data set that we're pushing back on people and so and we've heard during the day examples around the systems the games that that we see on the screen is the line of code learning or or is it adapting to the data and so it is finally the data itself like it was just mentioned that the data is if you have enough data then to a human the perception is that the system is capable of understanding I I I Define chatbots as three main components and we're mixing some of those now which is the first is the nlu capability of understanding what the human or the user is telling the the about then there is some logic around what the bot can do and finally there's what's returned to the user scripting text and and so if I write 800 lines of response based on different configurations of of nlu understanding it might seem to be a system that that can adapt but finally it was all scripted previously was trained for that so um and you're using some third party platforms like uh Google Google and Amazon and other bot building platforms to actually build and deploy Bots out to customers is that right yes there are there are some commercially available platforms like IBM Watson uh Microsoft Lew U api. and that's only for the understanding part of it so how how well do those do with limited data or do you need to pump in a lot of customer data to get good results I I would say those research organizations are are the most commercially mature offerings that can be found out there they don't require a ton of of training data to to spit out a decent uh understanding like a model that understands uh Dennis on that front of data so you're one of the few uh organizations out there that's actually tapped into the mind of these Bots can you say a little bit more about the kind of data you're collecting and what patterns you're seeing in the text I mean we're we're we're seeing we're seeing a ton of data and that's kind of what's what we find super exciting about um sitting where we are is is like we are seeing you know frankly like a lot of straightforward simple Bots right now and kind of to to answer your question before is in ter like I don't know how long it'll be until we see kind of true multi-turn uh conversations but we're seeing kind of glimmers of Hope where at the very least you know a lot of bots are are either responding to button pushes which is a very simple conversation or answering straightforward questions um but we're starting to see like mean remembering your previous question like saving State um so you don't have to just say you know give me a a black pair of genes in size large um you can say I want some genes the bot responds um and then can remember that those are very kind of you know small glimmers of hope that people are even trying to think about these things um but I would also push back and say like you know in terms of the the course of normal human conversation uh we have a lot of simple conversations as well and you know we we start off with ordering you know off of menu and that's very simple not not necessarily multi- turn conversation um but it can be and I think that that's what's exciting is we're we we have a billion billion or more people interfacing with this interface so when it gets better we don't have to tell them to use something else um youa on that point I think you were talking about vertical uh versus generic Bots that can answer any question um so when you talk about like actually putting a button with a specific text string on it like that's a pretty customized bot right yeah uh that you're talking about like can you say more about like do you think uh vertical Bots are going to be uh narrow use case Bots are going to be winning or or broader God bots so this is a very good question first of all I'm talking about from the natural language understanding perspective meaning you know can we solve this problem from a research point of view okay not from a rule based perspective um one of the problems we you know for example Siri is that Siri is a very generic bot okay a very generic personal assistant and it cannot answer very complicated questions for example if I'm saying to see have Siri I have a cell phone and I have a problem with my keypad every second word that I typ it doesn't it creates some kind of a problem so Siri won't give me an answer for that but the other end if I'm trying to be more specific for B specific one of the problems that I can you know get more domain specific data you know train my system maybe it can create some kind of you know um deep learning model some kind of let's let's call it a machine learning model that we solve it but one of the problems from the business perspective is that can I scale it up can I create a lot of you know different type of use cases that you know that will generate me enough you know critical mass of you know generating actual business on top of that and that's the real challenge so are are customers willing to pay for this kind of thing right now so you can take a bunch of data and turn it into a an interface that's so so here's the thing you know customers you know it's it's a really simple question customer you know the customer is thinking about is thinking okay am I going to save money or am I going to make money on of it okay it's no customers you know there are a lot of customers say oh I need to invest in Ai and there is a lot of a lot of hype about AI but you know eventually you know when you are talking with you know with with the business guys and they are doing the the officers because of course the data scien they say oh my God I want this type of Technology no these guys are making very tough business decisions so they don't care about the technology behind it okay you can put a small monkey that you know just just answer you know the the codes and it will be okay but eventually know if you are creating some kind of a solution that will give you some kind of a business opportunity they will going to willing to pay for that um so I think I mentioned uh got like vertical Bots uh one of the things that we like we've talked about on the uh the Bots podcast is um God Bots versus vertical Bots and by that what we mean is um if you're Google or uh Samsung or uh Microsoft you you have the device you have the OS that the bot that has a primary bot on it like Siri like Alexa sorry if this is recorded to everybody in the future who has an Alexa um sorry but not sorry I kind kind of funny trigger people's alexas so um anyway if you have those devices you're in a speal special position because your Bot is the one that people get used to interacting with right so bot Discovery is a very hard problem um and any anybody building one of these specific customer ser you know a vertical bot that I would use as an end consumer um would have to know to install it so I guess my question so like how how do you think you mentioned that you have a platform so how do you see these other vertical Bots coexisting with with Viv so it's a platform so developers can come to VI and teach VI whatever vertical they want to teach right um given it's user facing so it's actually has to be useful to people and not very small specific people right um so to Broad audience that would use there and in terms of very specific vertical Parts like business focus I think they can definitely coexist all right um so I know Kotana has right some Enterprise level things in which um they are collaborating with companies so develop Enterprise level BS and who users are not going to be using those ones these are people who are actually maybe Factory operators who want to operate their Factory using a natural language interface they would use that so this is like completely two different products and they can completely coexist in terms of user facing chat bars so let's assume we have a chat B that just answers weather questions right versus uh Apple Siri that can answer different questions I think I I'm biased but I think think like things that can answer more questions you would talk more to them right unless you start seeing errors in which you meant one thing but it went to another it thought that you're talking about hotels instead of weather for some reason right because it has to make that decision at the global level of what exactly you're talking about so unless you start seeing errors like that and you start getting irritated with it then you might be might go to chat bar things which are you know very specific would answer to your requests and you know the domain very well you know what use cases they have right um but given that if you're able to build something that is accurate enough I think a simp pressing a simple button on the phone is much easier than going to a simple thing we we saw this play out in in both the web and the mobile world right like in web you had SEO so you had like Google essentially the godbot that you mentioned um that shifted over to to new speed op to new speed optimization um and in in the world where you have you're staring at your Le um I think kind of Bot somehow teaching Alexa to optimize for your particular skill might be the next thing we're looking for okay so there's going to be a a bot race kind of like ano race beo or something yeah um beo yeah bot engine optimization I don't know and also and if I may and also the delegation we talked about B Siri Cortana these U main general purpose Bots that might not be able to interact with all pieces of software and they might need vertical or Niche specific natural language interfaces where they can route specific requests and get those fulfilled by a vertical bot but but uh summon that vertical bot and return back to the human user with a confirmation of the of the task that has been requested the the danger with of uh having a sort of handing off an experience another bot is that um a lot of these systems have different levels of sophistication in terms of uh how much of natural language they can really understand can they understand pronouns can they understand referring expressions and um you can see a lot of sort of whiplash or something as you transition from one system to another um and so it does seem like you would at least want some kind of common language um or common design patterns um that show up again and again within language well so Jason Jason is a common they thing language pattern so switching to English there might be a discovery mechanism where both Bots can Define what language they're talking might be able to understand what capabilities they have and attempt to perform a transaction potentially yeah yeah I think it's a little difficult to sort of you know declare that you know you handle referring Expressions while you handle pronouns and then sort of especially given that we don't even like linguists don't even agree on like what exactly is going on with a lot of these things and what things are different concepts and what art from a practical standpoint oh go ahead oh no I was just going to add that um to the conversation that one thing with the current systems um having a multi-threaded um dialogue with like Siri or cotana or any of these systems is really really hard right um both from the user perspective and from the system perspective you don't know what exactly the system remembers about all these different conversations and the system has a huge problem knowing what exactly you're talking about and how they're connected to each other so I think if people want to have a lot of it's if it happens that in future we realize that people want to have a lot of multier conversations and stuff then you have to come up with some kind of solution that might be made aable of them between them I mean that's hard that's hard in real life too right like multi thee conversations exactly so I don't think it's a huge I don't think it's a huge use case um but if it happens to be that way that oh you're booking five things at one time and then you want to go back to all of them yeah well that that's the power of AI and Bot so from the bot side that it could be booking and coordinating for example their uh x. and um Microsoft has a a meeting booking bot where it can know uh it can have this Godlike view to know everybody's schedule and get come up with a better answer um and it can optimize there there's other yeah so there is a promise of bots I think multi-threading better than humans um but I think there's a different point here which is like how do these Bots coexist so another point that was raised back on stage we're still in the b in the Primitive stage of bots where they're almost like a reflex action you know you you you hit your knee and your leg kicks so I say um you know weather to a bot and you know weather San Francisco and it spits back the weather right um but multi-turn dialogue is a little ways off similarly if you have Bots coordinating with other Bots they need to know like person they they'll need to have the contextual information that Google or Amazon would have and and so like that feels like a difficult question like if I ask a question when's my meeting it needs to know who you are right so like David you have you thought about you mentioned you worked on voice a bit like what do you think are the challenges with context and conversation um there are many uh so uh probably the biggest thing is um actually you know it's pretty easy to just hold on to everything they've ever said and every you know API query you've ever made holding on to the results the difficulty is in sort of figuring out what when they say something what they're actually referring to back in the um conversation and um that takes a lot of uh takes a lot of training data and a lot of sort of uh sophisticated models in order to be able to handle that especially um because language is such a sort of fluid and flexible sort of soft medium um yes you you've been working with voice as well like what do you think are the unique challenges there's different types of bots here I mean there's a chat bot which is a conversational text interface now they're enriched with graphical buttons but there's also voice which is you know uh specifically the OS spots tend to use voice quite a bit can you say more about the challenges see yeah so first of all speech recognition over the past 5 years you know has gone much much better around you know I can give you some numbers around 75% better in the world a reduction since 2011 but one of the problems that people think you know speech recognition is still an open problem know it's a very open problem because for example if I'm talking from a very close microphone I'm talking just for to my to my cell phone from very very close the recognition is quite good but if I'm talking in a very noisy environment or I want to put a microphone that will try to transcribe what I'm saying in that room the recognition collapses and when you are thinking about conversational interface so two problems first of all the speech recognition adds some propagated error to the text information and that is why one of the problems when you are trying to think about a can I integrate with Amazon Alexa or with Google Now one of the problem is that they are just giving you the Bas the best hypothesis meaning you know what is the what is the best answer that Alexa is thinking you have said and this is not good enough usually know when you are trying to think about and also if you're thinking about languages okay so okay so with if we are trying to create some kind of a chatboard you know creating a chatboard you know in English or in Spanish or in Chinese it's really easy because you have a lot of you know data over the web you know when you know the transfer learning is much easier than in speech so just as a reference right now there are 7,000 languages in the world Google supports 80 languages so we are talking about you know just more than 1% of the the world the world languages that we have that was interesting I saw I think Berkeley actually had to take down a bunch of their course material because it it wasn't um available um in other uh the audio hadn't been transcribed so this is another problem is like text to speech uh if you're impaired if you're hearing impaired how how if you're spending all this energy as a company building a voice spot how does how does that work for the hearing impaired right uh so there there's a lot of uh interface difficulties that you have to think about when you're building these things are are you so you're building these for companies now right correct can you say more like what are the types of things I kind of want to lead towards this question of will Bots go bus so a big question is are people paying for these things right correct yes so so are they and so what you can see is similar to the web in the '90s uh Mobility 2000s companies some have expertise in house where they they have dsk um language experts uh scriptors writers and some others might not and so that's when a Services organization can step in and assist definitely similar to how companies 10 years ago they contracted to create mobile applications they they need to enhance their software products and connect have interfaces for conversational features um and and the device the channel gets deluded it could be on the desktop it can be on web it can be on the mobile phone even specific devices like you said Alexa or Google home uh that are controlled by by the tech Giants so your bullish on conversational interfaces I would say Pete it's the same uh as we've seen in the last couple of decades web appeared and we start to see users having a facility to access information and transactions on the web then the facility of going to mobile devices having your phone with you 24/7 and and you know the statistics it's the first thing you check when you wake up the last thing you turn up and from there trying to solve transactions and get information through the mobile phone we're now having a conversation with natural language so it it comes to us humans naturally to be able to interact with software in natural language it could be text or it could be voice and so yes I on on my we're bullish on on voice and and conversational interfaces going forward they'll have different formats for different software products I I see you nodding a lot Dennis like you're seeing some real data like what are you seeing repeat engagement we're we're continuing to see like uh more users interact with our Bots than you know the previous month month on month we're seeing more and more people having longer conversations with all of our customers granted we you know we're growing as well as as a company as people discover us um but what's encouraging is that to see how many many people are actually continuing to speak with these Bots and have these interfaces um and unlike you know 20 years ago when I first started doing web there's a lot more people that you can access now that are available um through Facebook through you know kick line slack uh in Alexa Google home like these devices are are ever present and I think that that's that's no no small feat that an opportunity of an opportunity right to to make it more concrete like what are some examples of bots you've seen that seem to get some engagement or repeat engagement yeah I mean some some of the BTS right now that we're seeing are frankly you know very very straightforward uh domain specific Bots like the weather Bots we're seeing that's a very very simple thing um there's it's kind of like there's some Bots like like swelly is kind of a hot or notot bot which is a very a very like simple like but very engaging experience for um a lot of their users and and I think those are those are some of the early successes we're seeing in the space um and then I'm also I like I love the uni clot uh you can actually have a conversation with it and and shop shop their you know their entire product catalog through a conversational interface um but there's there's there's definitely uh people running experiments and and playing with things you know this is I'm I'm super excited it's like when MIT put their coffee pot online right like they put their coffee pot online a bot like no this that was from the web the web days right yeah the old mit coffee pot um there's there's there's fun things like that um you know in the bot world um yeah you're seeing adoption uh I think it to kids who are going to grow up in next 10 years this is going to become the default thing um I see my nephew and niece that hooked on to their Alexa even though half of the time they doesn't really understand them because they're still learning to speak um but they are so used to just talking to a device that I could have never imagined before yeah I mean conversation is the first interface that we learn as humans right um so to have the ability to interface information with this very natural interface is is super exciting um and we as technologists are very fluent in like SQL and typing um and I think more of our world's population will be more familiar with accessing information in this way and that's exciting to me how many people out there in the audience the show of hands are working on conversational interfaces or bots in some form okay that's 20% 30% okay um so it might be good to uh turn this over to the audience and get some audience questions okay uh question yeah one okay yeah so I think a lot of the talk you guys given it's actually a lot I I feel like it's a lot more towards the consumer side um I just wanted to hear if you how you guys feel about Enterprise level how Bots can be applicable there well I mean we have Salesforce uh sponsoring the conference with the reinstein product you can see how that is a conversational interface to a ton of ML and training on the back um but I think that that's one example of of a a business conversational interface in in corporate um David you had some thoughts on Enterprise versus consumer uh sure um so it's you know customer service is a very big expense for many companies and uh and we already see a very high rate of Automation in these uh systems and a lot of uh dissatisfaction with the current systems but there's such a cost-saving measure that people um still invest in them and so I believe that as we sort of make uh sort of more fluent dialogue systems um more available and more more effective that we'll actually see a lot of uptake in that area um replacing sort of this more typical ivr systems that you get when you call your bank uh hi I was hoping the panel could address the question of about some conversational interfaces voice interfaces in automobiles I mean these things were already rolling computers and more and more we going to be uh tasked less with driving them and more with figuring out what to do while we're sitting in the car while it drives so yeah I mean I think voice is is the the right interface for for an uh context where the the user doesn't have availability to tap or click on on on it on the device um and uh and I think that skill keeps on growing with new generations of because I'm I'm so excited to see the tech Giants opening up their car platforms as well to create ecosystems and orchestrating these ecosystems around content functionality application um at a level approved by the automakers and at a level approved by the orchestrator and so carplay Android auto and others combined with voice interfaces I think great will it be the car though or will it just be the phone that's on your dash well either one right I mean I think I think it it's it's a fun mental exercise to kind of wonder if the voice inter interes will beat the self-driving interfaces right so um like which one which one's going to win out right cuz at at the point of which they're fully autonomous vehicles we don't necessarily need voice to to hold our attention anymore um sounds like yeah because I've been doing voice interface for automobile so I know some of these problems um first of all you know the problem with the automobile that it's a noisy environment okay and you have what we call you know you have a lot of noise coming from the eyes side from The Mott okay so it's you know as you said as I said you know there is a challenge in terms of the voice interface okay because it's not you know a very St clean environment and it also you know it has a lot of challenges from the acoustic perspective because you are talking in some kind of you know a closed acoustical thing so it's really promising you need to think about the use cases okay uh as for the mobile or as for you know if you are going to put a microphone it's from the technical perspective you know we need to have a microphone either it's you know either the processing is being done on the mobile phone or are they the the car not really important the main challenge is the speech recognition stuff cool we probably have time for one more question here anybody else okay I'll just turn it back over to you yeah okay um this has been really interesting um I think uh the the jury is still out it it sounds like the consensus at least backstage was uh that Bots the the AI behind Bots is not going anywhere um and conversational interfaces it feels like a natural way to interact and so we're going to keep doing that maybe like closing thoughts we've got another like few minutes here to wrap up each want to give like a 30 second take um sure so I think that where these interfaces are ultimately going to show their real value is in uh problem solving and dealing with things that are not better served by guies and that um and that sort of long term you'll see these systems as an augmentation on top of goys um except it may be the case of like automobiles um where yeah there's a limit to how much you can put on the HUD um and so I think that that's going to be where these systems get really interesting is when they're able to reason about multiple constraints and orchestrate um uh sort of multiple tasks across multiple domains to help you sort of get things done um without you having to Think Through the steps yourself very carefully what about you is what do you think I think there is the well I've been working on conversational a for the past 16 years so of course I'm I'm really in favor of you know this this stuff will catch up sometime um because of you know the the past five years that the technology has been evolving there is a strong promise about it I think that in terms of the use cases you know um we need to solve real world problems and to show the customers all the the Enterprise customers or the end users how do I make their life better so we need to concentrate about on the use cases that are really promising and not know just very simple use cases do you think Enterprise over consumer in the near term well if you talk if you're talking about a startup company so a startup company of course you need to Gin revenues or a set or a set of customers I think that you know Enterprise is more promising because from a startup company if you talking about you know a b2c you need to create a very large you know set of users which might be difficult you know because you have a lot of competition from the big guys my my thoughts um if you work on on a software product then the process of defining the one or two features that are best served by a conversation interface that's the main exercise to come up with the top one or two features that allow your users to engage further with with the product I I would recommend that small exercise as a means to starting to get into the conversation space yeah I guess um kind of what mandrew was mentioning earlier uh as we move from web to mobile and now you know with so many people interfacing with these platforms now the opportunity is that when you have a ubiquitous interface that is more accessible to more of the world's population that kind of a little more convenience a little more ease in usage is going to result in huge effects that affect you know billions of people so that's that's fun yeah I think conversational interface is going to be the future um and very very soon we are not going to have the talk about whether it's going to go Bust or is going to succeed um it will become more useful uh we'll still increase we'll keep increasing our expectations so AI would never be solved right uh we'll if it can perform something now we'll always have the expectation that it will perform something else tomorrow so we'll have to keep improving the system keep teaching the system uh how to perform different tasks and it has to be a system that can be taught very well by the users as well so it can learn from users what do they prefer in um perform tasks in future great well let's uh give our panel a [Applause] hand thank you all