SF Text: Jay Wacker, Quora Ontology @Groupon

SF Text: Jay Wacker, Quora Ontology @Groupon

Recording: SF Text: Jay Wacker, Quora Ontology @Groupon

thank you uh very much uh for having me um this is actually my uh first uh presentation outside of uh physics believe it or not um so um so hopefully I got the level right um a I was asked to give a talk about the uh topic ontology and what I want to convey to you is that we're still in the process of really scoping it out I've been part of qua now for about nine months um and it was a user generated ontology before it was in you know a reasonable shape and now we're trying to really uh systematize it and go forward and make it into a world-class antology um I can't really talk a lot about the products that um are being that that are you that used ontology but I'll will talk about the structure and some of the philosophy behind it um and the constraints that we have so um I imagine a lot of you know what quora is it's about sharing and growing the world's uh knowledge through question and answer so uh you have someone who has a question and they ask it on quora um they write a question and then we distribute it to an answer that um one answer finally you know one possible answer finally writes an answer and then that we distribute that to readers and the reader reads the question and has follow-up questions and the cycle goes on and on and on um and it you know really works uh wonderful um and it's basically premised on the idea that if you know the answer to the question it's really easy to write on authoritative answer if you don't know the answer it's actually much harder so for instance if you watch the Academy Awards the other uh um a couple weeks ago and you saw JK Simmons he didn't talk at all about the entire production crew of whiplash didn't even think the D the directors or anyone else he just spoke about his family um and if you were like me you thought hm that's a little bit weird um and you wondered how did uh people involved with the production of whiplash think about uh JK Simmons not thanking them um that seems a little weird um a couple days later um one of the producers of whiplash Cooper Samuelson I think he's that guy there um happened to be on qu and came across this question and uh wrote a basically a narration of what what happened JK told us that all that he would be using the Oscar speech to make a personal statement um he had plenty of opportunities to uh thank us beforehand and everyone who worked on the film uh thought his speech was great was lovely especially my mother um I don't know if his mother actually worked on the uh film or not I don't think so but he's since written actually about um over a dozen answers on the production of Whiplash and there's a lot of like inside information there now um so it's great um you know if you know if I you know you look at that question say who the heck would uh be able to answer that you could speculate boy that's really out of form for the Oscars maybe they told beforehand but how do they feel well we know now so um that's basically the the organic nature of quora and topics which form the Bas of the topic ontology the entities of the topic ontology um are applied to questions and only questions um we don't tag answers separately from uh questions um and it's really actually important that we tag get the topics applied to the questions at the time that they are written so you have the asker and they write the question and we must get the topics on at that point in time um before we distribute this uh question to potential answers and this is actually a really challenging problem that you know we're constantly developing on um and that's because we only have one to two sentences to identify what the subject of the question is and what and how we should identify it and identify who the best potential answerers are um and I I tell you it's uh quite challenging even as a human to identify a lot of these uh uh what the subject of a lot of these questions are um and so um what we do is that in addition to any sort of of backend systems that we uh use to identify questions uh algorith identify topics algorithmically we also have the ability to have users um correct our suggestions and this helps us improve the system uh going forward and additionally um after the question goes live and gets asked and is out there in the wild um other people can come in modify the question and they can also modify the topics and so the topics get evolved and this actually helps redistribute the question to better and better potential answerers um so topics play a critical role in that getting it to the person who actually knows the answer um um who potentially can know the answer author authoritatively now on the other hand after um after answers are written topics play a critical role in distributing these answers to potential readers you know writing a uh you know an answer is great but you really want to have a red by you know tens of thousands hundreds of thousands of people um and to do that you have to know who are potentially interested in this and this is where topics um also come in um and so um after the answer comes in um we dist distribute these answers out to uh readers and so what you see is that the topics provide you know really a Lynch pin in the cycle of asking answering and learning um on quora so um they really are an essential part of the the whole system so I'm going to first spend a few minutes talking about uh what our conventions are for topics and you should feel free to uh um stop me at any point in time so I'm going to do just a basic classification of topics and this is um a lot of this comes about due to just the historical usage of topics and what was one of the hardest things to actually realize is that there were actually two different classes of topics um on qu that were that had been created and used historically um there were what I will call Standard topics and what these how these topics were being used were to identify the subjects of the question um sort of intuitively what I would have uh how I would have used it so for instance what is the best type of Chinese restaurant the subject of that question is Chinese restaurants um and we should note that it's not uh we don't um want people tagging uh applying topics to um as potential answers so Sichuan restaurants or funon restaurants um or Chinatown are not appropriate topics to be applying to this question even though those are related Concepts they are not actually the subject of the question um and so this is how we want the topics to be uh used and of course there can be multiple topics so why didn't JK Simmons think anyone involved with Whiplash um so that's JK Simmons is obviously one of the subjects of the question the uh 2015 Academy Awards is implicitly in there because that was when the the this occurred and Whiplash 2014 movie was obviously also a critical uh subject of this question and so um so people who you know know and have experience with these topics may be able to uh answer this question or provide uh interesting context for it so these are just sort of standard topics yes so you may about to answer this if uh but you mentioned that you don't want potential answers to be uh tags to to be tagged as as topics for questions um is there any kind of enforcements or testing that you were able to do to to help that along yes the question um so the the answer is is is there are there any ways that we actually enforce this uh um uh condition that we don't want potential answers as uh topics applied to questions and there are systems both the community and other systems based um both algorithmic and various other varieties that I can't speak too much about yep here you classify into multiple multiple topics here the probabilities or totally binary right now we we treat each topic as um being uh uh equal um so the question is do we treat each one of these topics um equally we um we have uh some information um about which topics we think are more relevant for the question but we don't actually use that at this point um I think we've experimented with it but uh right now basically a topic is a topic is a topic on a question can you touch on um how you go about these topics kind of use or I can't speak about any of that I'm sorry and then the other question is is Best Supporting after not a topic just as um of we um we could have had a best supporting actor topic I don't think it actually exists on quora it's something that uh you know it's part of the topic ontology that's missing uh best supporting actor 2015 Academy Awards um is a very uh appropriate topic that could have been applied I don't believe it exists follow that you have lat topics topics that you don't really know what is yet but they are kind of still classified or are grouped I'm sorry are there latent topics latent topics kind of like a like a topics you can't get the label on it but but still you see that you know five different questions seems to be about that um we do have uh some indications that we don't uh actually SE we don't actually use that information at this point um are there any user defined topics like T all these topics are the almost all the topics are user defined something they actually put in tags no so um the topics are essentially the hashtags of quora so you at mention them and they are treated as entities um and so um and they're linked and uh all the uh um information is around them are coated great so these are standard topics the other class which has many different examples are what we've come to call question tag topics and this identifies the type of question it is um and this is actually um realizing that there are these two distinct categories has really led to a lot of uh um Improvement in how we think about topics at quora so how do I get rid of bed bugs um it's possible to apply it's a howto question um and people had created topics of this variety identifying the type of question it is um and um you also applied bed rugs um as a possible topic but this is obviously a complete completely different type of entity than than bed bugs right it's a asking for it's describing the class of question um that that it is uh um that the uh that the question belongs to um as another example why is air travel so expensive well that could be a um tagged with question that contains assumption is it really that expensive is you know um and that's identifying the type of question it is some property of the question but not the subject of the question and so these are actually very different entities and the reason why we separate these two types out um between standard topics and these question tag topics is that sub the subjects of the question is where people have expertise and interest howto questions you don't really develop expertise in answering how-to questions or question that contain assumptions if you answered 50 questions that contain assumption whether you know the answer to the 51st question is uh is essentially unrelated you know I asked answered uh 50 questions about air travel all that contained assumptions the next one is about tigers um okay um I know nothing about tigers and so it's really the the type of question is um the identifying the type of question is really quite distinct from the subjects of the question yep are the two groups topic with label the in question um who decides what topic so the community decides I have final say um and these were actually pretty well identified and separated out in different types of uh parts of the uh cor taxonomy of topics um beforehand I collected them and identified them as a different ontological set so they were kind of intermingled in but sort of grouped off in some ways and people realized there was something different about them um and so identifying them in a systematic way and moving them off is uh um um is you know something that uh has happened and I I decided it but they were more or less understood by the community to be different um and a lot of times how people are using topics is implicitly as a question tag um so as a question tag rather than as a subject of the question and so that's one of the challenges that we have to communicate to um in our systems um identifying the systems is when historic Topic application has actually not been about the subject of the question but it's been about the the type of question that it is involved with so um this we have a big uh um Corpus of knowledge and that we have to be uh um identifying these multiple different ways of using the same topic um to make sure that we're going forward in the right way um and these have um these types of topics have actually been pretty heavily used but we're scaling them back because they really don't uh serve our system serve serve the purpose of topics very well um so this is one of the sort of sort of highest level level um like most course graining of the uh type system of topics um at quora um another thing to appreciate about quora is that uh topics go down to a very fine detail um finer detail the most Wikipedia articles it's more like the subsections and and Sub subsections of Wikipedia articles is sort of like the level of topics that we want so restaurants in San Francisco um is a uh you know um is a perfectly fine topic but there are restaurants in North Beach um and then you can even get Italian restaurants in North Beach as a potential topic um that that has can have real meaningful content um if you get a restaurant owner who's owned a restaurant in North Beach for you know 30 years and writes about extensively about the history of Italian restaurants in North Beach um what you know what all the issues of you know owning restaurants in North Beach the interaction with uh China could be really interesting content there and it we don't want to have it uh you know conflated with uh um say the very high level topic uh restaurants in San Francisco there's really this separate very small niche of information so we really want to get topics down to very small fine levels of detail um we have a lot of topics and the basic uh um point of view um of the companies that we barely started to create them at this point so we're going to be creating more topics going into the future um down to very fine levels of detail because people really have experience and knowledge of very fine levels of detail and a lot of the best questions are about very small areas and people like to read about that so it's useful for us uh so I'm wondering if you could talk a little bit about how when do you have metrics for how detailed the topic is and there can be some confusion sometimes between uh how deep an ontology a topic is and and uh you know whether onology only only helps you you know that the coverage neontology is uniform and so there's some tension and understanding the difference um so we do have uh some metrics for that um you know the the topics that have been around for a while uh the size of the topic is actually you know the amount that it's been used and followed is often a good indicator but there are a lot of new topics and so um understanding like where it falls in the taxonomy and how it relates to its uh topic and how those parent topics are uh um how big those parent topics are some ways that we actually infer these uh the actual uh um how large and how large of an area knowledge given topic is um and yeah and so we can uh we have ways of inferring that um yeah so when I look at the uh at the restaurants in N restaurants in North Beach example you have up there clearly there's a there's you know one hierarchy around the location of restaurant and another concern which is the kind of restaurant um how do you how do you manage these two different I'll actually speak about this uh um at the end as the uh one uh one of the forward-looking things that we're building um building out and it's one of the few forward looking things I can actually speak about so um um and I won't be able to give all the details obviously but uh some basic Notions of it um okay so the the rules about what are appropriate topics um that should be topics at the end sorry um so topics should be nouns um and generally plural nouns so houses not house topic should be Atomic which means that we don't take unions of distinct ideas so New York and Miami and Puerto Rico um you know individually fine topics but we don't want to have a topic that is both all three of them or this case two of them um so we should be uh um reticent of that a lot of these are have actually been created uh historically so um we have to divide them up um topics should also refer to single ideas and um so that that you can actually speak about them in a faithful manner so that people actually know what um um there uh that we're speaking about um and this is also very important for when we start classifying topics because if you're speaking about many different Notions of the of the something with the same name how can you possibly get it into to uh taxonomy in the right place so like the Captain America creative franchise you know that is something that overarches you know everything involving Captain America from the comic books to the movies to the little uh toy figures um to the character itself uh to the various every other entity associated with that creative franchise um the movie is obviously something quite different um the character is different um and so um we need to dis ambigu these different ideas um if we want to get these things uh put in the right place in the taxonomy and also to know what sort of uh properties these topics actually have um because the creative franchise is quite different than the uh Captain America uh Toy uh soldier Okay so with that as just sort of the basic background of what the uh uh topic system is um um I'll go forward and talk about the actual um ontology of of the topics and I should say that theology I mean two different thing two two separate things I mean both the tonomy of the topics and also the type system and the information we keep around each topic um so there are parent child relationships and this is the one type of topic to topic relationship that we have we only have one at core at this point um and it's a directed relationship from um a topic that is uh bigger um and totally encompasses a smaller topic um and so you have physics and you have quantum physics that is uh under it uh perhaps not even as a direct child but as a it could be even further down the graph but schematically um the quantum physics is clearly a um a a subset of physics and so the questions on quantum physics are a subset of the questions on physics and and so with these parent child relationships we can form a taxonomy of topics um so um our topic graph is rooted um in uh something that we call major topics it's just uh doesn't have any content and it's just there to be uh the root of the graph and then we uh Branch out to 18 levels at the first level into sort of the big major areas of human thought science engineering technology food and cooking this is partially driven by um you know how the topics are the topics under them are related and also some editorial judgment about how we plan on uh how how we actually think people um conceive of these topics do you publish your ony um it's visible in product um so um the the ontology is visible and editable in product by any user so it's Community Driven uh but we don't publish it um so if you want to go and inspect it you can there's no AP for it um and we strongly discourage scraping so um so um at um level one there are 18 uh topics at level two we're getting down to sort more of a um more reasonable topics and we have 110 elements uh different topics at this level um and then it roughly grows geometrically for six to seven levels under it um sort of spreading out um at a reasonable Pace at at every Point um at each level and we've U made sure that through level three things more or less uh look sensible um and that uh and that they refer to similar levels of uh spec uh specificity um and so yeah um but this is just the beginning we're planning we're probably planning on having it be you know 12 levels deep in all honesty once you actually build out the full uh Quon ology yes where would your Italian restaurant in San Francisco in this um why don't I come back to where the Italian restaurant in San Francisco would fall because um I'll get back get to it uh shortly so um most importantly um it's a graph not a tree so the taxonomy um you can you can have multiple parents um not just a single parent so this is actually very important because there are many Concepts that are synthetic you know the they're the intersection of two topics um so physics and biology it's not a perfect example but biophysics is um the inter uh the intersection of the two um and so this allows us to actually um address some of the natures about when you have multiple a concept that inherits from multiple parents where where it comes from and then the question is what are the rules around that so um it's obviously um a cyclic you don't shouldn't be your own parent or own descendant um but we also have rules that you shouldn't be your own aunt or Uncle um and so this is uh um I don't know if i' ever seen this before but it's like if you have a parent Topic in blue here um and you have a topic Mr yellow here and M green um green should not contain yellow under it or blue shouldn't so that's a u mistake um the scoping of the topic either the intersection of the um of the yellow with the green is Meaningful or it isn't if it's isn't meaningful um then it shouldn't be under there if it is Meaningful it should have a separate name um because you mean something different than the thing outside of the um outside of the uh u in just that intersects with the blue and not the green similarly um it goes down the tree so further descendant should be distinct from each other um you can be your own cousin or other types of relationship um there's no scoping problem so in the example of biophysics you have biology and physics that descend from science and their intersection is uh biophysics yes so it seems to follow that a given topic must be at a given a single level um no we refer to the level of a topic as the shortest path to the root topic um and in particular uh like restaurants um are both a concept around Cuisine and cooking and food but it's also a business and it isn't and businesses are a a level two topic as well and so when you get down to restaurants it's deeper um in the in the business side of the tree so so there's exception so you you don't need to be the same you just can't be a direct descendant of this sort so you can you can be on different levels um so you can be your own first doesn't once removed um but you can't be your own uncle or great uncle or great great aunt um so on and so forth under different branches yeah exactly so this is one of the the rules um and other ways we identify problems as if um all the descendants of one topic are coming in from a one one topic so you can just that's often a sign that there's some Mis Place parent child relationship um so there are lots of tools that we can use to make sure that the scoping of topics are appropriate one of the biggest problems that we had um from the organic ontology was actually academic topics um which I actually was a little bit surprised by because we had academics who spent a lot of time and love on the um on the uh on this part of the uh ontology and there was a real problem with just the conception of how it was organized and this is I think dude goes back to the academic huus uh so there's chemistry and the way that it was organized was that chemical substances were a child of chemistry um and then all things chemical fell under uh the chemical substances so CO2 and then under CO2 there was atmospheric carbon emissions and then there was climate change somewhere under there um what was even worse is that there was cultural studies then there was American culture under that and then Americans under that so Angelina Jolie yeah so um and this is like um and you can obviously see uh in retrospect uh what the uh what the uh um solution is you have chemistry which inherits from it's a physical science and it's it studies chemical substances so it's more of the intersection between a physical substances and uh chemical substances and the physical sciences um and then all things chemicals descend from here chemistry not pick up CO2 and all the children of CO2 and all the possible uses of carbon dioxide um so that's you know and otherwise you'd have biology as a you know as a descendant of chemistry and and you get into really sort of absurd uh relationships in physics it was the whole physical Universe was under physics um including uh you know every person alive which is obviously very meaningful um so by Des scoping uh uh the the academic topics so that chemistry is the study of chemical substances um and is a physical science you can actually uh um clarify this and make sure that the descendants of chemistry and all these acemic topics actually make sense um because you'd really want that to be true um so that you can really go down multiple steps and make sure that you're still in the same rough area of knowledge um and this was one of the Hu biggest problems uh and I should say one of the most confusing ones was history because we don't really distinguish history from the historical events around the topic we use that the term actually um very interchangeably so the history of France does that belong under history or um and then all the historical events of France which can come down to the present so like uh Nicolas s uh could be uh under the history of France which is under history and that really is an overs scoping of what history actually studies so you have to separate out the historical events of a country or an area or some area of Knowledge from the uh from the his the study of the those events um and so that huge mess okay um the type system uh there's another uh slice of the standard topics that is uh um very important um one is whether it's an abstract concept restaurants universities physics companies or whether it's an example of the topic specific realization it's sometimes called um so Moma Fuko noodle bar is a restaurant obviously um Noma alenia um you know this is a very you know important because you know abstract Concepts if you identify them should not have locations for instance and you shouldn't be able to say review an abstract concept or um do lots of things that abstract Concepts that you it's perfectly acceptable to do to examples of Concepts it's perfectly um appropriate to review Noma and say it's a really awesome restaurant um so um this um part of the type system hadn't been developed and it's being um rolled out as we speak uh slowly but steadily um and so actually this um if you go into a lot of uh uh topics some of them have some abstract Concepts had uh um locations associated with them due to some mistake a while back and if we had had this type system in place we could have prevented that okay the most important uh um type of uh relationship that is sort of special that goes beyond just sort of a notion of parent child relationship is an is a relationship um and we really want to separate this from a standard parent child relationship in in quora uh for several reasons that I'll discuss um so obvious you know Gary Dano is a restaurant in San Francisco Frisco um restaurant in San Francisco is an abstract concept it doesn't really it's you know just this notion of what a restaurant entails Gary Eno you can go there see the black out windows and think wow I W really want to eat there um and so this type of relationship um we did not create a separate uh type of of uh parent child relationship instead what we did is that we created a contain a structural topic um so that San Francisco in San Francisco has a keyw in front of it specific restaurants in San Francisco which Gary Dano is a specific restaurant in San Francisco um at some point in the future we might migrate this so that this composite structure um is a uh you know is just a different type of relationship but we're still building out the this type portion of the type system yep so um if you have emotions and the you can have happiness you can can have basically have things which canot have specific things right so the restaurant does restaurant know that can have specific can I I think that what we'll talk about is just after this slide um so another reason to do this is that you know this is a relationship instead of being uh um sort of uh one to a few it's really one to many like there can be an arbitrarily large number of child topics um arising from is a relation relationship um there's no limit in principle um and so if you want to actually think about the topic ontology is being navigable um and um you know it really makes sense to sort of separate off these type of relationships which are very different than say the restaurant scene in San Francisco which is a distinct concept that it should be a child of uh San Francisco of restaurants in San Francisco and obviously these are not parallel Concepts in terms of their level of specificity um one is about specific restaurants one is about the restaurant scene and it's another abstract concept so having this intermediate step where we can uh um intermediate topic where we can separate off these relationship is uh as is very useful and also makes it clear that these are parallel relationships and again at some point in the future we might just modify it so that there's a separate uh uh set a new type of relationship but is a relationship is not the only type of uh relationship that you need to um consider is a kind of relationship is a another type which is um abstract concept to abstract concept which I think is what you um were referring to um so um there's restaurants in San Francisco and there are Chinese restaurants in San Francisco Chinese restaurants are still a um um still an abstract concept it's not a restaurant um it's but it's a specific type of restaurant in San Francisco so we use the key the the key phrase specific types of um to denote when you have is a kind of relationship and so this is uh where um how we denote these different types of relationship that are sort of one to many um we don't encode as parent child relationships um many to many relationships um so you know we don't want relationships like like this in uh in the topic ontology where they're just kind of back and forth and so like actors and films actors participate in many films films have many actors did the the actor was part of the creation process of the film but should they be the parent of it or should they be the child of it um it's really not clear depends on your point of view um sometime um and so you could really end up with problems with scoping and also it's basically impossible for an actor or the film to be completely encompassed in either one of them so like uh um Chris Hemsworth um should um you know being under uh Thor um the movie doesn't make any sense from a scoping point of view there are a lot of other things that that Chris hemworth has done um and Chris H and questions about all questions about the movie Thor the production value the lighting of it um the CGI really shouldn't flow up through Chris Hemsworth and so these types of relationships we need to uh just ex out from parent child relationships these are related Concepts um and they um should be stored in some other part of the uh knowledge base um similarly startup and VCS startups typically have many Venture capitals who fund them and VCS um fund many startups and uh someone had actually uh um really systematically gone through and encoded this relationship as parent child relationships throughout the uh startup World um which meant that uh um basically every startup went through Finance um as their uh parent which really led to some bizarre sort of scoping issues so uh y you think of these abstract as classes and then is a sort instantiation of that class in a sense or I I think that I mean this is all category Theory at the end of the day and I think objector programming is a uh is a uh is an example of you know of category Theory um so I think that these are relationships these are the mappings between them in a category um so yes I think that is a um an example of this type of relationship uh that is very general throughout mathematics um and in fact I as a physicist I always disain category Theory I thought it was a not all that interesting but in fact I think the next thing I talk about is the the first time I've actually found a little bit of use for it which is location which was uh the question how do we encode location in all these fine grain topics so um this is in progress um you know there are a lot of details about exactly how we um do this in practice um in terms of just making qu still feel uh organic but the structure of it is very clear so restaurants in San FR San Francisco is clearly a composite topic created from both restaurants and San Francisco and this is part of a general class of of topics where you have some locatable concept restaurants universities hotels um you know parks and location um so San Francisco New South Wales X and provance um they're all separate uh um you know um you can take the the product of these two structures um and just create all these are all valid Concepts and then the question is how do we organize these and the answer is is that there's a natural notion of parentage uh uh arriving from this uh product of of different trees so you can think of about restaurants as being part of a tree of businesses um and tree of uh locatable Concepts in fact a a graph of them um and you can think of uh Palo Alto as part of a our classification of locations and so when you ask what is the parent of restaurants in San Francisco well it's essentially a derivation you take the parent of restaurants times pal Alto and plus the restaurants restaurants times the parents of Palo Alto okay and both of them are parents or many of them are so let's say that restaurants the parents of restaurants is a specific type of business uh service business paloalto is a specific City in Sano County um and it's also a specific City in San Francisco Bay Area um however we choose to organize locations that um this applies to and it can have multiple parents and so when we perform this product of these two uh these two different uh uh um trees and add them up you find that the parents of restaurants in paloalto are specific types of businesses in paloalto specific types of service businesses in palalo restaurants in specific cities of San Matel County and restaurants in specific uh cities of uh the San Francisco Bay Area yeah so uh if I'm following application of the first part of your slide there where you uh if you will and then the parents trying to find the parents of restaurants in San Francisco on one side and the par and then how it was constructed on the other side sure should be constructed on the other side so it seems like in order to be able to do that there have to be some notion that restaurant and was a compound yes topic and that it has some components do do you have a model for the the potential components or different so there are there are types of topics that are um uh that are locatable topics and there are types of topics that are um locations and in principle all products are feasible um are feasible um uh topics to create uh product topics to create okay um how we actually build out all those topics is um is is the main trick because you don't want to build out you know um you know like I don't know Peruvian restaurants in in uh South centr I don't know where some some place that doesn't have Peruvian restaurants so that's that's the main trick is how you actually scope appropriately and truncate these topics um at an appropriate point of view and it's pretty easy to come up with good her ristics on how you do that um and I can't go into the real details of it but it's if you follow your nose in terms of like the feel that you want it to have it comes out pretty uh in a pretty straightforward manner okay I'm just finishing up right now um so we have the rest of the type system um is here academic Fields professional Fields uh professional uh professional uh careers um companies products and services locatable topics has a location um events recurring events um and then we also have some ones that uh still need to be rolled out so for some reason we didn't have a way of labeling something as a location um that's it's already need to just uh create the that and uh implement it living person um a person just and uh um and activities are all things that we're planning on doing these are all different ontological entities and you can imagine many different products of associated with each one of these uh classes of uh these different types what is the difference between Lo and has a location um locatable does not have a location so what is the difference between locatable and has a location so locatable is a class of uh Concepts whose specific um realizations whose examples isas have locations Okay so so restaurants doesn't have a location um Gary Dano has a location so um so the so the uh Isa um um versions of these locatable topics are should have locations okay and location and is a location is that that is the primary element about them so Death Valley should be separated from Death Valley National Park so the location of Death Valley National Park is Death Valley um national parks and Death Valley and Death Valley National Park um there so Death Valley National Park is a national park um and its location is Death Valley um that's actually not encoded in the uh it's it's it's only encoded via this uh product structure that I discussed um here um it's through this chain that we actually encode yeah and so it eventually gets up to Death Valley when the parent of this is nil or goes up to Major top um that's when it uh becomes Death Valley okay okay any further questions about the type system I can't really speak out all the different ways uh we can and are using it um okay so the Outlook is that the topic system is a critical um part of quora for getting the whole um question and answer process to work efficiently and well um this topic anology is still being built out um we have a large base of topics but we're only at the beginning um part of the challenge is that we need to standardize this organic structure um that the community aspect of cor is very important to us um and so we we can't just wipe the Slate clean and start over we want to you know keep the community uh um aspect of it so that the way that people have used uh Concepts uh in the past is still faithful yet it's standardized across the entire topic base um and that's necessary to make it predictable reliable and ultimately you know really usable in an Al you know an algorithmic sense um we're still developing the type system and what this really will allow us to do is be a lot smarter in how we use uh topics and there are lots of cool things coming uh using all this all these different uh type the entire type system okay thank you very much I have a a follow question there so no you love that excitement this very inspiring and this High experior presentation here that you really really really like it so question to you here just to follow what takes the physicist to get so excited about Poli and this whole topic um so why why am I excited about this um I think it's cool I love quora so first of all I was a longtime user um I've always loved mathematical structures and knowledge um and you know I I know a lot about lots of different topics and so it's a natural thing and so actually understanding what this different what the actual structure is um is really cool and figuring out how we actually use all this structures is really what gets me excited so that we can actually be smart about like what we do um and um there's so many like obvious things that be done once we have this uh um implemented across the board and so that's what gets me excited just wondering how in practice how you use this and how is this integrated like is it would you store this in like a relational database or would you store some other kind of store um you know so how do we store this um I believe it's a relational database you know it's wonderful because I don't need to deal with any of the infrastructure on the back end uh being a physicist that's uh they were they've been very good to me in that way not to uh have to worry about uh too much of the uh the details uh everything works uh beautifully uh from the uh the the topic system and uh uh in terms of the performance m in the the issues at this point um and I've been told that uh none of the issues here are going to provide any uh performance issues is it like the is the ontology generated like daily or is it real time um so the ontology can be changed in product by anyone obviously we lock down certain things that are possible vandalism but uh um but anyone so it's visible in line and when you make a change in anology It Happens Live um and uh so if you change the name of a topic if you change the the type system of the topic um almost everything is updated uh on the fly so how do you evaluate or validate those changes when you make them um how do we evaluate the changes um so one of the biggest problems is uh making sure that everything is scoped properly so that when when you when you want to have questions flow up to parents um so that you can actually find you ask a very specific question you want to start looking up the tree to find people who you may not have anyone at that level of expertise but you can flow up the the graph and making sure that actually makes sense and has the right feel and that the engagement is good and it's really easy to have um things that are off in it that doesn't that that just all the metrics just uh go to hell what interface UI interface or tool do you use to like VI this yes yes uh um it's it's I I've looked at various uh um ways of like representing it making pretty pictures it's way too large for any of that okay can you some um you can look publicly I don't like to say that number um but it's it's it's large um and uh it's you know you really need to deal with either like skims of topics or like certain parts of the graph um at which point you know you know I I found it's just as easy to uh come up with a a standardized analysis towhere yes we have not settled on that yet um yeah yeah um yeah that's work in progress um and we we haven't even decided what the uh what the potential benefits is of such things um of having it manually uh um like manually cating such a uh knowledge base I'm sure there are some but like how we would actually want to use it which would dictate even the scoping of it um so this reminds me some of the uhhuh at I yes I'm the only one that's uh um I've uh I've looked at it I've been influenced by many systems out there and um many academic books on this um both on the uh mathematical side um as far as I've been able to tell we the quora topic onology is not a copy of the the the systems that were developed by the community are not a mirror image of any um system that is out there um as far as I I can tell um and so we're trying make things that are standardized and make sense um and it's been heavily influenced by um a lot of the different uh systems out there um but I um as far as I know we do not we're not a copy of any uh exact system um but I'm always you know oftentimes good ideas are uh you know are attractors so um we may have uh um um converged on something isomorphic thank you how do you identify users as experts um oh I can't uh say about that but people write on certain write lots of questions on certain topics they also provide biographies on their topics um and so we uh we uh can infer yeah yeah um write good questions that get lots of upvotes uh on a certain topic and you will become an expert um we will identify you other question what makes like you just s of mentioned it before that something good anology something that makes sense to you but like me we we do so whenever we like uh so creating the products around this we have to make sure that uh that uh that when we start adding in these features and using these features in product I mean that it you know that it makes sense and it's at scale you know this is not like a um you know that that it's not like we're like having people interact directly with this this is you know sort of at the heart of the system and so when we start putting it in it's actually very difficult to debug so we can just watch what happens uh to various uh levels when people when we start using these additional features and then figure out what's wrong diagnosing uh what happens when it's wrong and and figure out why things went right um so it's uh um it's you know it's that that is very challenging but usually what it boils down to is that why the heck is this topic under here that makes no sense so um so it's so uh yeah and a lot of times that actually it's it's actually very uh difficult because you know you look at it and you say oh that makes sense but then you actually have to realize all all the ways that the topic could be ambiguous and all the different ways it's being used and so it's actually very difficult to to uh really dis fix that problem um is that disambiguate the topic so you can identify when there's ambiguities but actually figuring out how to uh to resolve that what the proper resolution is requires some uh brain work yes how often do you wake up in the morning and realize something um oh every day yes there there um we we're we're constantly improving things um and uh and yes uh the ant the the the taxonomy is not finished um how often do I wake up and realize there's a whole new type system or type uh or relationship type that's happening much less frequently now so um a