Text By the Bay 2015: Nikita Ivanov, Human Curated Linguistics - technology for Cognitive Analytics
foreign thanks everybody for coming in my name is Nikita Ivanov I'm a CTO company called great game systems I'm also a technology advisor to a company called Data lingua technology which I'm going to be talking today and uh I want to talk today about basically two things cognitive Analytics why and how why is it important and what data link with this specifically to actually Implement that technology it's pretty cool stuff and we'll talk about a human creative Linguistics as the technology behind that why we talk about a cognitive analytics and I don't want to give you the entire business spill on this but it's kind of an interesting aspect um a couple years ago I've seen the Gartner basically made very interesting statement I think it applies to all of you guys that the only people today that have effective access to the data in organizations other people that don't have any questions of Their Own think about it it's a data scientist data analyst anybody else people who have no questions of their own they only exist in organization to find answers to other people's questions I think about it for a second it's a very profound statement that's how we basically developed the entire bi in data analytics systems for the last 35 years we developed them for a very small group of people in organization they typically have no questions of Their Own and people that do have questions whether they have no tools and no ability to access the data so what else happened in the last maybe five seven years is a huge growth in SAS applications if anybody remember 15 20 years ago all about business data was in sap in Oracle or PeopleSoft systems like this today literally every company small startup to mid-sized company would have at least half a dozen different SAS systems where we keep our business out Salesforce with CRM Google analytics for web AdWords platform Google Merchant platform Twitter Facebook LinkedIn for AdWords workday for HR benefits QuickBooks of finance jira a significant project management list goes on and on any company today would have literally half a dozen different SAS systems that are completely incompatible I will challenge anybody you can find me one person who knows both Salesforce in Marketo for example they don't exist although there's two major systems that's used constantly in every business replace marketer with HubSpot problem against even more complex on and on and on so basically what happened today is the corporate business data is really trapped in silos and those are the true silos they're incompatible the data models incompatible query language and compatible like in Salesforce so you have to know SQL to query it in Google analytics you have to understand data warehousing it has dimensions and metrics so basically people don't really have access to it today except for the few and those that do process is fairly complex and and cost this is the basically what's happening today business users I believe all the data scientists typically in algorithm to spoke to a larger bank here in the valley the larger Bank multi-billionaire organization they have about 12 data scientists for entire Bank that's what people those 12 people have all the tools and necessary to actually access to that and they have thousands upon thousands of people that basically have to go to those 12 to really you know ask any type of questions so go to this they do some programming or some data science you know a couple of days later got your answer what's also interesting and if you talk to any analyst they will tell you the absolute majority of the business questions are not life and death issues that require your immediate attention it's typically you know simple question that people have a hunches during the meetings during the meetings with you know the teams the customers most of the questions don't warrant you know going to some special Department to ask them to produce some report wait for a couple of days and get your answer most of the questions we have are ad hoc at exploratory we want to basically see work on a customer meeting we're a meeting with our sales team what happened we just launched the AdWords campaign in New York did we had an in a pull-up of the website in the last couple of weeks typically questions like this don't get answered immediately because literally if you don't have it like a dashboard to build already you don't get that really answer that so cognitive analytics is the upcoming term that basically defines a a product or a process by which anybody in the organization can ask any questions about business data just by using natural language the basic way to think about think about as a series for bi in dat Analytics you know 10 years ago we didn't have city of Cortana Google Now any of the Technologies we had to type and navigate all complex interfaces today we can simply talk or type most of the time and ask our questions and really get answered I always bring this you know um important compatible look back in early early 90s search of a network was the engineering problem think about it who would write in right mind would be searching for some document on a local network all Engineers would do right it was engineering task fast forward in the 15 20 years back and forward I mean anybody kindergarten in his grandparents can search the entire globe of information that we're talking a few words in Google exactly the same transition with happening with the behind analytics as we speak this by the way uh this is a screenshot by the way in the end I'm going to show you the live live demo of the system it's pretty cool by the way but this is a live shot of what data label looks like if you really want to ask questions about a data in your business this is as simple as this it's actually a very Google ask if you ask me there's literally a text field in the button there's nothing else you need to do that's a that's a fascinating difference between everything you guys know look at every data science platform any Dynamic platform any bi platform for that matter it always have much more complicated interface with you know complex UI in the same everything else here you can literally talk or you can type and basically ask practically any question today and get a real-time answer and we'll talk about what's behind that by the way this is the type of results you're getting so a very simple interface in the back end on the front end to ask the question but once you get the answers the answers can be fairly sophisticated you get information back by just asking simple questions it's a type of questions you can ask think about this you can actually ask something very generic like some of the top 10 most effective City if you lost 90 days and if you're connected to Google Analytics you'll get an answer now take a look at this I don't really know what active means right I don't really specify what active means all I have basically know that I have a website and there's notion of activity in the website and every system will treat it differently for example Google analytics has sessions in new users and users another entities in it but all I have to do is to ask this very generic question to get exact answer back I don't need to know what specific data model is behind Google Knowledge we also support Salesforce and some you know some limited form look at this question for example um over here what is the correlation between new users and opportunities created for each state pretty cool question just imagine how much it will take you today to answer the same question so the question is do you have any correlation between new users and a new opportunities that we just created in the sales force maybe you know people come to our website and they're buying something right away there have been nine questions today I challenged anybody to produce a result in less than a couple hours you have to go to Google analytics create report massage it export to a spreadsheet have to go to Salesforce if you have a report lock if you don't program in SQL get the report export the spreadsheet normalize dates formatting enjoy them together then you get your result back and that's what you have to basically know two systems it's a multi-hour process if you have two people working on it and how easy to get two people work in the organization in system like data language that takes us a few seconds you literally just ask this question and we'll get back to the sponsor so other questions can be fairly complex we're not talking about simple ones you can actually ask a very detailed question and you'll get the response back like this for example you can ask you know um give me updated by states where essentially the use of numbers increased 20 months over months for the last two months it's a pretty sophisticated question again to answer this question Google analytics doesn't support these type of questions at all by the way you have to run two reports for multiple months in manually in a spreadsheet join them here since we have a complex SQL post process and we will understand now this is like an important I present a lot about this technology in lately and there is a fascinating properties about the natural language when it's applied to bi in data analytics first of all it provides radical simplification that's pretty obvious right what can be more simple than something you and I already know think about this natural language is the only one interface between you and the data that you don't have to learn you already know it if you can speak at least in English you already know it it's very fascinating properties zero learning curve zero zero better adoption think about this you hire the new person in your company she walks in the office first time first step first minute she can already make daily driving decision because you already know how she already knows how to you know extract data how to ask questions it's very disruptive ability in this way it enables millions of people literally the bi is not new in daily life is not new what it does it opens up an existing business process to entirely new class of users all of a sudden not only those 12 data scientists can actually access the data anybody in the organization with the proper permissions and security can answer practically any data it's a very interesting ability what's also interesting is this it is ideal interface for all our new upcoming devices for mobile and version wearable if anybody know Apple watch there's no keyboard on it right there's no keyboard on most of our devices going forward we're going to be talking to them at least typing you know at least accepting the text input if systems not designed today with acceptance of the speech or text or the input they'll be pretty much outdated pretty soon look at our GPS in the car half of them already support speech input so we're going to begin going this Direction look at the Cortana from from Microsoft look at Google Now how they move Enterprise space look at Amazon it's literally getting there pretty quickly so the speech we need people will be pretty much the key one but this is probably what's the interesting the natural language is the only common uniform interface to all data sources think about this what's common between Salesforce HubSpot and a workday dancer is nothing there's nothing in common different data models different interfaces different accounts different approaches different everything the only coming out between them is that you can express your question in a normal language and if you can in the system understands that's the only commonality there I can ask questions you know what's my total revenue for the last year on the East Coast that's a good question I can ask what's my average benefit cost for the last quarter that's a good question for work day I mean I can ask HubSpot to know what's my top funnel numbers for my leads right I can ask those questions and I have no idea how to use any of the systems but I can ask those questions and if the system would understand me that will be a breakthrough because I could use all of this data for my own needs so by now you're probably asking how the hell you guys do this because it sounds pretty fantastic so that's basically I want to stop here for 10-15 minutes to really come in I'll give you the rundown of how do we arrive to the system we have today so the project started you know about two years ago and originally you know as almost anybody else and it'll be fast we decided to go through a kind of break you know through computation computational approach computational linguistic approach obviously if you know anything about NLP uh pragmas and languages don't compute you know you can compute you know a strict grammar sentences pretty quickly the minute you have a context which is pragmas in the language that basically provides you this semantic context they don't compute uh it's impossible to compute it properly or translate them into the SIM card in Intermediate Language so we hit the wall pretty quickly and then we kind of swung all the way different directions saying okay we're going to use the Deep learning stuff neural networks the problem with the whole deep learning you know situation is that but many folks don't realize it's not an exact thing it only can give you probabilistic answer it in it has built in precision so it's acceptable for a lot of use cases so for example if you do tweet a sentiment or any kind of sentiment analysis or computerized Vision or categorization imprecision is allowed it doesn't really matter if your Twitter Twitter if it is 65 or 66 positive right it only matters that it goes in right direction and kind of in relation to the group values but if I'm asking you what my revenue for the last quarter I need an answer to the pinion that's why for example IBM Watson cannot be used in any of the bi systems like this because IBM Watson's deep Learning System it can give you know what your revenue for the last quarter was about you know two million dollars with 95 probability what do I use of it what do I do with this information you know what's very interesting that for the bi in data analytics for some of them especially for the corporate business data imprecision is not allowed so essentially what we were faced a couple of years ago was that we cannot use any of the existing tooling for Technologies at all computational breakthrough you know kind of work that doesn't work uh deep loading doesn't really work as well so we start searching basically and we came up in a pretty interesting study how our brain works and how we analyze the sentences let me give you an example imagine ask you a question um again back to our web analytics example what are the top 10 most active pages on my website imagine how many different ways whether humans can ask this question between ourselves can you show me what are can you list me can you display to me can you email me most active most visited most viewed most touched Pages web pages HTML Pages website Pages for the last whatever weeks months days from this day to this day it's a permutation really go in in tens of thousands permutations although it's exactly the same question and our brain is wired in such a way that we never spend a second trying to compute all this we immediately see the a core semantic patterns in this question so for example we immediately understand that this question is about active Pages for this data range period if you say top 10 we know it just give us a top 10 only so we immediately you know narrow down to this key semantical components of a question immediately describing all the stop words to stop sentences so stop you know soft sentences that was pretty interesting and we thought you know can we build the same algorithm and we did so it's actually what we've done we've developed a a process by which we take any sentence that you basically either say to the system or enter through typing and we reduce that sentence to a a theoretically minimal set of semantical dnas typical reduction goes by order of things and Times by order of four once we have that semantical sequence of semantical dnas as we call now here's the key process here we look at our system and see have we seen that sequence already if we've seen it before that's that means we already have the answer for it we have a query for it so we basically do automatic substitution and return result back to the user if we haven't seen that sequence before we try to come up with the automatic answer for it but this is the key we give it to the human being to double check it or correct it now think about this the human correction is not new quite a few companies isn't it but that process gives us two unique properties first we never introduced error into the system because there's not a single question in a system that has been automatically answered without a human interaction each question at certain point of time was automatically answered but checked by a human analyst and corrected as well second of all not only about the correctness it gives ability to process a completely free form add restricted language the way the first system as far as I know in a world today that you can make a grammar mistakes you can misspell things you don't have to follow any specific grammar you know rules obviously if we can understand at all some kind of bearish we will we will not answer this question but you don't have to spell things properly you don't have to follow a grammar you can use the same language that you and I speak talk about think about this on the Skype instant messaging none of you follow an exact grammar we don't do this as humans right you know we just start the way we like in using some lingual you do some shortcuts in the grammar as long as we as a humans can understand what's going on we typically have this how we interact right if you look back on the history of NLP systems and we did a pretty extensive study there most of the NLP systems with this kind of you know question ask ability fail because they were asking people to follow an exact grammar an exact set of words but if you're following this people basically don't like it because I might as well just learn a particular interface and just follow an exact words an exact grammar that I need to follow for it to be convertible to let's say SQL this is the first system because of this human interaction in in a loop where you can ask pretty much any type of question with any type of you know limitations or shortcuts in grammar or misspelling women contextual confusion if you will and it will be possibly understood by the system now I get immediately the questions you know well you can use human for each question wouldn't this scale as a business or technology it will first of all this Loop is real time in real time it takes about to involve a human to double check a question takes about five to ten seconds and that's a know-how with this company that they build it in it doesn't take overnight it doesn't take an hour it doesn't take a minute typically it takes just a few seconds for a human being to double check or make a correction to the answer so for me from user perspective they don't see a difference they basically see a question being answered right away with pretty cool answers what we also do over here on a on a once the question is being asked in real time in offline mode we have a pretty deep supervised learning algorithm and let me just give you a couple of ideas of what we do essentially there imagine that we have answered on the two small questions we have this you know two DNA sequences two sequences of dnas already answered and we know they're correct those two sequences for example we can take a one sequence from Google analytics and once it comes from Salesforce and learn how to answer more complex questions in a full automatic way and we can do that because we know that each part is correct we don't introduce any errors in that algorithm it's pretty advanced stuff what really boils down to is that the system has tremendous self-learning capabilities with every question that hits the human being we we're not only learning how to answer the old permutation of that question which can be in tens of thousands fermentation the same question we're also learning how to answer more and more combined questions from those small tokens we actually just answer that so we the company is pretty early right now but the models show that has a at least exponential growth in terms of the um in terms of the self-loading capability essentially with every question that human being answers more and more questions after sometimes thousands more automatically now being answered in the system so it's really cool is that the calculations show that probably within nine to 12 months system can saturate its knowledge about any particular data source which means science will have about a year of work it doesn't mean it doesn't need to have humans all the questions already been answered and correctly in the system and that's very unique property of that design that we don't have to keep the human beings on data source forever they can be navigated source and so on yes yeah this is the snapshot of the interface that basically exists in the back end it looks very sophisticated but most of the time humans just have to look at what's been what was the question and what actually is performed as automatics analysis and literally most of the time just say yes or no if the question is is automatically answered and it's an incorrect then basically goes to the fixes something that's where you can spend in all those 15-20 seconds that's for sure because if you're just looking at it and say yes no it's a two second soon write that boom boom if you need to correct it or extend it then yes it can take you know up to half a minute still it's a fairly low a fairly fast interface because half a million versus hours it takes today is is a huge progress so this is this is actually uh I think it's my last slide I think and it makes it a couple more but this is basically a snapshot of what we see on the back end of the system so back end is fairly complicated a lot of signs that pretty deep stuff uh and uh that's the interface the data analyst from the back end sees in real time so just a few words about a whole system design um this is kind of lesson you think this is more interesting uh we're fully in memory system uh we're based entirely in memory architecture for a speed we use a flash at night this is actually the company and the project that I started many years ago um it's a fully memory system so we run Italian now my own pretty large glass of computers uh we use basic Stanford NLP and wordnet for all the basic NLP you know uh mechanics you know we obviously don't do any part of speech or lamb or stem but we just borrow it from NLP instead of NLP we developed a lot on top of that we have very complex for example you know there was a couple questions about stop awards that they didn't have fairly complex Dynamic staff work mechanisms but obviously we don't do stop words nobody does stop words everybody does stop sentences stopwatch like five of them so stop sentences actually dynamically create them Based on data source and we've been pretty smart about it in this way we can actually deduce stop sentences very effectively not just by words and we have you know fairly sophisticated uh analysis based on stand for NLP so we Scala back-end entirely we'll have Scala on the front end you're going to see the demo in front of basically very typical it's Anglo GS node.js I'll use cardo for all the mobile appliances ahead of the storage pretty good stack on the market automation but I'm not going to spend too much time here so I think this is the last slide and let me actually show you some demo stuff it's kind of cool to look at it I think I have plenty of time about 10 minutes okay so I want to show you guys see it perfect so this actually live system it's not a demo it's a live system where it's companies in private better right now uh if you want to sign up just go ahead and sign up you can basically get a access to it if you like so let me go ahead and sign in over here I remember my password properly all right so this is a very simple interface by the way the Simplicity this interface is on purpose you know we pay the company pays a lot of attention to uh to the ux and UI type of things here keep in mind this can be used not by data scientists it's going to be used by average Rank and file business users and that's you know why interface is very clean very simple very not threatening I'm trying to keep it this way so and it's pretty simple and that we have a you know typical toolbar the interesting part is here baskets your inbox where your questions come in and all your questions that you're waiting for for example or you answered before uh they're here you can look at it so let's go ahead and ask remember the question we basically been talking about like in the top 10 cities right we can go ahead and ask this question um one thing you notice immediately that the text box here is pretty it's pretty smart it has a lot of models built in so it predicts what you're trying to ask so you don't have to type a lot of things very important thing is that for example for geographies everybody misspells names of cities and countries in state like Mississippi for me not American Border I'll misspell it every time so for this thing since that we have you know pretty cool article for example if you want to ask for like Moscow it has a Moscow if you want to know how San Francisco spelled you can find a lot of sand something else but let's go ahead and ask the question about um top 10 most active Stadium so we can say show me the top 10 most active cities for the californium and we can pick this one for the last three months for example something like this and that's it just go ahead Android what you're going to get back is a pretty uh nice looking and pretty uh simple results set that has a lot of information in it see on the screen so on top of it you have you know a lot of you know stickiness you know and we'll call the kind of hard creating those features if you will you can give us a feedback you can see which data source it came from but then you can see basically your question in a little bit different shape and form we're actually showing you how we understood what's happening here so show me was detected as a stockboard by the way this is pretty cool if you just you know detect stopwatch as a word you will never mark show as a step word it's a verb it's a very strong verb you will never actually you will never give it as a stop word but if you do it a little bit more smartly you can you can basically detect this show me literally stop stop sentence here and we do pretty good in our analysis there so we actually basically show you the top 10 results that limit uh the series actually is the Google executive because we know we're talking about googolics over there the California here will be geography location and the last three months has been just data range so you can do a lot of different things you can export it you can share it I can customize this whole data you know many ways as you like it uh you can basically show this things over there if you like just to make sure we can show them and uh and basically everything else is pretty obvious here we have the charting that's actually here as well and you can play with the strut as much as you like a little bit different formatting here since the order you have the geography information in this question we also showing you the um the mapping view of the same data that you basically got now pretty cool right take a look at this we just asked this question right it's a very simple question and by the way typical this question came in the dashboard for Google is not really interesting move we'll look something more interesting but you get a fairly interesting result back fairly workable and it's fairly knowledgeable and it's easy to consume what it's in there now if you go back remember I told you that basically if you ask the question in one shape or form you cannot basically change a lot of these properties without ever going to a human being right so we can check we can change a lot on these questions so for example we can change you know say 20 we can say show me please uh active C is for let's say not C let's pick one say that's Nevada it's not going to be just a couple of cities there right and we can say you know let's say for the two quarters so we changed the question somewhat added words again we just can click and it will be automatically answered because this question is exactly the same as the previous one it doesn't matter if we change the geography remember I talk about this reduction to the key key symmetrical values that's exactly what's Happening Here difference between California and Nevada isn't isn't isn't important it's just a geography location right there so it's it's really getting the reducing some symmetrical value so is the the uh the dead range so is the top 20 by the way look at this place police also got to be a stopwatch and as you know we immediately understood that and it didn't affect our previous cash value of discretion so we got basically the same back kind of the same type of question same type of response back what's really cool by the way is that in this system the ability to detect the ability not to give positive false positive is extremely important because when we talk about the bi in deadlines for the business data we cannot we don't want to guess if we write we can only should give you a result back if we know 100 is correct how about this let's just come up with a completely arbitrary question for web analytics that looks right grammatically correct but we don't have a data set for it I don't know something uh let's look for correlation um between let's say website sessions and let's say China state holidays yes it did well active basically it didn't affect any kind of in a token but it was not a stopwatch it was an active word it was used in analysis we just didn't we just don't show like a point of speech you know tokens here we're showing only something that's actually available to you no no it's actually not great no no that will be Mistake by the way so let's go ahead and ask you know a is there a correlation uh between I don't know uh website sessions and China state holidays sometime here now this question is pretty cool by the way I know for a fact we don't we don't we haven't really seen this before so it will actually go through entire Loop right now now what's really cool about this question is that it looks right it's grammatically correct it actually makes Point makes sense for the data for the Google analytics data source we have probably in the future we're going to be able to answer it but I know for a fact we don't have this data so let's go ahead and see what's going to happen here so what happens here you can see the cycling over here it basically shows that it's going to take a little bit longer to answer this question maybe like five to ten seconds but will get actually a pretty good answer here that unsupported data source now what you probably haven't noticed here is that this whole question right now went through entire human review did anybody notice the big delay or anything else two or three seconds but this question right now on the back end of data lingua went through automatic system it discovered that you know I don't really know about State China holidays it went through human data analyst who looked at it right now as we spoke and press the button or whatever it did and actually confronted yeah we don't have it a response Got Back three to five seconds all it took for you as a user to actually have that you know to this interaction and this is the answer is absolutely guaranteed and that's the key value behind that we don't guess it you know we're not trying to predict anything else if we don't know for sure we'll give it to the human being he will double check that yes there's no data for China state holidays you get the answer back which is the correct answer for data source we don't have a data source to generate State how it is and you get all of this is just a little late in just a few seconds pretty cool stuff now let me yes well we support you know right now the data linguist supports only specific data sets like Google analytics and Salesforce and they're working some of the others as well adding this information is not trivial it's not like you know we just have to give them some data set or some spreadsheet we have to integrate that typically our belief that it will take about several months to add a new data source so for example if you want to add more care to it or a HubSpot it will take us about two three four months to do that so adding the state holidays is actually just it's not a really data source it's more common reference data for us that could be a lot easier we can actually do it pretty quickly like within weeks and we probably will do it we actually already have a technology work in the back end at weather information you know flight information they think they're actually available if you go like you know this Yahoo Marketplace for the data stores they have plenty there but that's just the reference that if you're talking about the entire data source that has its own data model its own in your life like HubSpot or anything else or workday it will take longer it's a pro it's a project for us because we're able to add the whole entire processing on the back end we have to train out that analyst we have to have a UI for it in the back end so it's involved process still two months yes actually it's coming that's one reason that you know right now we don't really have a full support for Salesforce because unlike anything else Salesforce has this you know words of adding the custom Fields some count time custom tables custom use into the custom systems so it's hard going to be support that probably in the first version we will not support anything other than custom tables but you know Salesforce is fairly involved process product but we'll get there it's we actually already have it it's just another production level description yes for the casting Fields if we don't if you cannot mind the names of fields they have to provide it that's going to be the part of the sign up process so if you sign up for the Salesforce we're going to ignore casting Fields by default if you want to provide them to us you have to basically spell out exactly what they mean because there's nowhere for us to digest that yes breakfast and then analysts in the back end so we'll be able to see it happens yeah so it's actually it's a good question in in this system data analysts do not know anything specific about a customer and it goes for a privacy issues for security issues as well for scalability issues we cannot like you know have people know it about thousands of customers we're hoping to get in someday so some of the questions may not be answered correctly an idea is that you know we're not pretending that you know we're saying not correct it's the wrong way not optimally we're not going to give you the wrong data set we may not give you enough or may not give you or may give you too much maybe you're only asking for sessions for somehow I'm going to give you 20 other fields as well so that's why on this interface you know you have all ability to you know filter this data your ability to Pivot this data to add columns change data data range automatically and whatnot so our expectation that people will not only expectation it's actually what we learned from kind of user Discovery is that people will use this system to get quickly from the hunch to the first data set actually I had a conversation with a couple of users you know a couple of potentials and they said Nikita We're not gonna just go back and say you know what type sort by something or P would buy something we don't do that we want to have initial question like this get us back the response and then we're going to work on this screen with whatever else we want to do we're going to be able to sort anything else if we want to do something more expect export to express it move to SAS most of Tableau work further so this whole sci-fi now you know Vision that people will just pick up my microphone like they say sort by something that's not going to work special Enterprise maybe not right now maybe in 20 years they will but not right now so most of the time people expect to type something quickly get quick response because that usually takes days to get something from question to a actual data set but once you have a data set they want to work in this in this interface then I want to go back to typing text or saying something so what you know what the company is trying to do right now is basically to bulk up on this interface because the basic you know technology is there we can understand the language they want to bulk up on this so that people can actually add columns you know add data change data click on something you know drill down you know drill up all those different things so questions right now or you can catch me I'll be here throughout the whole day yes exactly no we're not going to support anytime soon unstructured because then you have to basically uh look at your database and explain it entirely to us which is pretty hard sales cycle so right now we only support not only instruction but the well-understood data services and there's plenty of them there's plenty of business to have there eventually we will get into it but it's it's just not only not it's hard for us it's hard for you as a customer you can imagine you have you know two thousand tables and go ahead explain to us what the hell they mean it's you know it's it's a project for customers and we don't expect that people will do it but for a structured data we can do it out of mic and that's the beauty of that you can literally it all it takes are five minutes to sign up once we open it up and you're gonna be able to ask questions about your website literally by speaking to a computer pretty cool stuff yes right now just the one in company probably going to start with one or two with the Gated sign up process it's a startup right so it's going to stop very small but the beauty of this again if you look back at this you know approach of ACL the the whole bit of ACL is that it has tremendous self-loading capability so it learns extremely fast with every question that human being touches it probably learns about up to what's 10 000 different variations of the same question it's pretty cool pretty cool ratio of human touch versus learning and the coolest thing we don't have errors introduced that's what really boggles many different systems that you know you have to constantly sift through the erroneous or just the flat wrong answers the now system doesn't really happen because there is a this human touch and by the way there's obviously more than one human touch we do double checking in the back end so we do all the right things all right guys thanks a lot