Text By the Bay 2015: Matthew Drescher, Incentivized Question and Answer Data
Recording: Text By the Bay 2015: Matthew Drescher, Incentivized Question and Answer Data
big thanks to Alexi for inviting us today um I'm here with poshley we're um we understand that text by the Bay has a big uh question and answer component um and poshley is a business that is completely driven by questions and answers um so I'm going to tell you a little bit about how we are um approaching this um sort of large scale um creation of questions and um the Gathering of the information which is actually uh kind of a a primal dual relationship um which was a little surprising to me um so why do I have the Mechanical Turk up here um it's not because the previous presenter was talking about Chess uh but we sort of decided um to take an approach um a very a very humanistic approach so we actually have humans writers um domain specific experts writing the questions um because this is the best way to get our users engaged they know how to engage the users um and then at the same time because we're creating the content um it makes sense we sort of know a priority how to extract um the implicit meaning out of it um so that's a little bit of the background all right so as I said we're a um a uh sorry um we're a uh consumer intelligence company um and what that means is is we're basically trying to like I said engage consumers and through the content from that engagement um provide a value to uh brand professionals um that will give them some insight into these people so I'll give you some concrete examples of of what we're doing um so what we sort of have if you go to poshley decom we have these giveaways uh which are are at this point it's primarily cosmetic the Cosmetic industry um we have these products uh you know like makeup uh hair products nail polish Etc um and you can sort of like the idea is it's as similar to a neelon uh survey where neelen actually will pay you to fill out the survey well we give you a chance to win a prize um and then you'll answer some questions um so we try to make the questions fun engaging um and we sort of have a lot going on with how how we sort of generate the questions um it's it's based on a sort of an ontolog ontological graphical model um which I'll get into a little bit um and at the end of the day we want the insights that we extract from this whole process to be engaging and to power you know um classical statistical methods and you know some machine learning so one example I came across recently of this whole notion of implicit data versus explicit data is you know a data ninja you'll see if you go for you know if you're if you're looking for job ads for data scientists a lot of places say yeah are you a data ninja you know um but to actually train a computer to understand that a data ninja means actually data scientist is sort of non-trivial so our approach is to sort of just say hey let's just change it over by by hand in the analog of the Cosmetic industry um if that makes any sense so this is kind of what you see if you log on to poshley um these are the these are some uh products that sort of are happening you know a few hours ago when I made this slide um these are giveaways if you if you if you click on one you'll get the opportunity to um answer some questions um we have about we have over a thousand questions in our system um and the questions that are actually selected for you are determined you know algorithmically um and we try to sort of fill in I I'll get to we have an ontological uh sort of graph of of the different sort of human body parts uh that we try to and we try to find like a covering of this graph so that it um so that it gives us more complete information and it can happen algorithmically um so these are some examples uh you know what's your skin type um do you watch The Bachelor so like some of them are kind of just for engagement and this is sort of the interesting part to me um you know so if your skin type depends like what does that mean in terms of a an actual data point uh you know or or do you watch The Bachelor well that doesn't seem to have anything to do with uh you know the Cosmetic industry or or yourself as a as a beauty participant um so but this is in there because because our domain specific experts they understand uh they understand the consumer and they know how to engage them so it's important and and we have logic that you know tells you how to extract the information out of this um so here's another one um are you into nail art um you know some of these ones I wouldn't imagine like if I'm if I'm dressing up in costume or celebrating holiday I like it uh so that's some of these are kind of surprising how to extract meaning so the thing is that our our our subject matter experts like I hope I've shown at least a little bit they know how to engage um they know they know the audience um and we sort of the challenge is we just kind of like want to unleash that we want to just allow them to create content um and not really have to be constrained too much about um are what's going on under underneath the surfaces how we're you know Translating that into features Etc so this sort of the challenge in building the system um so here's an example of the sort of ontological structure I was mentioning um we have uh you can see you know we have body uh which contains fingernails lips um armpit you know um and and as we're like constructing questions we try to fill as much of this thing as we can um to get you know to get data on an individual so the logic also we have other logic to say okay if if we know you're bald don't ask questions about you know hair um you know if you don't wear makeup then you kind of shouldn't you know we won't ask you hardly any questions but um so there's a lot going on in the question generation so then the other side of it is like okay we allow our subject matter experts to sort of create whatever they want and and you know we encourage them to be creative as creative as possible um but then how the hell do we interpret it um you know and and without sort of like building a deep blue model you know to interpret anything that they come up with right so it's kind of a challenge and we sort of took a more Mechanical Turk approach to it at least initially um so so the the the challenge is sort of uh we have very high um velocity data so sometimes we you know on People magazine we get 20 answers a second so we have to actually build a system that can scale and at the same time be manipulatable um by our our experts so sometimes you know the information that we're extracting from these answers it changes based on you know where the question shows up in the survey um based on the priority there's a lot happening and it's happening at a large scale so you know do you do yoga what what insights can we obtain from that um maybe we decide okay that means you you live in an urban area because yoga is kind of big and then U maybe later they say no actually it means you actually live in the suburbs or you know we think you're athletic and then later on we decide actually no it means you're not athletic or you know whatever and does it even have anything to do with with cosmetics in the first place so that can change as well um also it because of that it might be a low priority question that only shows up um for relatively few people so how do we need that to sort of have meaning um so we need to sort of map these things to actual implicit insights um so to sort of facilitate that uh you know we want everything to be immutable um we're kind of like building the everything is a sparse Vector it's sort of the data structure of of of choice for kind of all of this stuff because we can just update uh the dimensions kind of ad hoc um and you know it it it kind of works out um so that's what's going on there and uh at the end of the day you know we we have these sort of ad hoc dashboards where we can present you with um data based on all these insights that are implicitly gathered from uh questions that are sort of creatively created um and I think that's it so I probably finished a little early so if you have questions happy uh yeah sure so we we want to sort of cover um so we sort of look at the right now we're kind of focused on Cosmetics obviously um and we we have this concept of an ontology representing the human body and we like to kind of cover as much of that as possible so if we get one question that's about you know fingernails then we don't want the next one to be about you know nail strength like so so we want to cover the different parts to build a profile um and then we can sort of say okay you know similar users similar people you know red hair if you have red hair and strong nails and then we can sort of like fill in the other gaps with cluster methods so we try to sort of have a covering of as much of the graph as possible if that makes sense question is um so there's also this concept of prioritization which is sort of moving around based on all of our data um so we might have a lot of data on you know hair care at this point and we need more data on on something else so that priority changes so the question the questions are are changing based on priority and also this sort of graph covering more or less that yes that's part of it you pick the product and then initi the questions are are related so if you pick like yeah no it's okay thanks okay thanks I