Text By the Bay 2015: Ashok Venkatesan, Near-Realtime Webpage Recommendations “One at a time”
Good afternoon everyone my name is Ashok I am a research engineer at Sanlappa method personalization team basically data analysis video engineer on machine learning and engineering basically we experiment just give an overview just comment sign up add interest and they Raman content fm d content is called and wicket user submissions from over d web so they don't they don't crawl the other verb but they crawl certain websites but not content comes from user submissions so they and when they recommend episode devices so you can dumb on to d up and lot of place golas recommendation they also server lot of things around it which basically facilitate and understanding user behavior and everything so they can book n you can bookmark you can organize and you can share quick look at recommendation so what is recommendation basically it is matching user with content so it's basically 4 points you can doing this again and again you understand the user understand d continent Recommend and get feedback on different kinds of content, it's text, image, video, news, and different language, and problems in challenge, basically it's understanding the user, quality, like, quality, meaning, can I be an expert, I'm ... Diversity vs familiarity and new TV vs fighting so only things are afraid so do you character dem who do you balance demand managing item supply demand and also item channel item supply demand is basically you want to be show date and if someone is falling some really in topic and if you want to be you want to make sure date you have enough content in the topic that is supply demand channel where you don't want to pigeon hole people in you think over again so you want fresh stuff you come up so you need to make sure date cup learning other model animation learning model apply everything sold and refresh it's there is no one of friends and they have planted business related rules that challenge and think is one rocket at a time those like miniature recommendation system recommendation time fox on the recommendation completely and user dos not get you pick and Choose right I mean he can't see watch going to come next m and again a thoughts also inside think a if it of us quotadaar Netflix and Amazon which d people they deal with cute content it is products products coming with curation a if it is movie this movie is coming with accurate a they are dealing with automatic feature extraction basically it's purely automatic don't show already seen content yes they they try not to repeat anything style whether I have dips some times date come in d from text in b put in maybe r blog and in b posted by and newspaper something like date d from news in also correct forward constantly changing product use cases I mean offset in r iOS us me totally change date bill screw up which d baby you stay on top up date always so this as of d problems and challenges an high level overview of ho d architecture lux it's some basically five parts you it the yellow lines the yellow arrows just an Paint the data flow so they come from injecting you into recommendation so you in just content and you do some content analysis the cold start model decides whether you know it's good and you're going to the recommendation rhythm and one seat publisher the offline computations take the offline computations as well as the online computations start updating it so it is sort of an Lambda architecture and an off set of girls you have the recognition engine which finally decides what needs to be put together today a recommendation going a little deeper into the recommendation engine the recommendation engine contains basically a cluster of strategies and at other points of time based on your user kid you are and more choose different strategies and every strategy has multiple methods this is open the hybrid approach and literature where you are doing content based training and collaborate filtering together so method can be collapsing it bill drink can be Content based drink n h method know ho you start h method what you filter and it gives it to d mixer mixer know ho you both d method which d method Okay so my couple things every happening 1s di and consumers this is in real time n this is one up d base in which one of d real time aspects of recommendation n where n they give power d empower and consumer and date is one when one one listener was looking at certain sets of you just signals you d empower it you know decide whether the cash needs you be replaced and note so just primitively it me trip saying date in this area you user wants this something like date right so something date is note an very of and talk about this d idea recommender system and this is sort of d motivation but same they want more said indicator i bill [ __ ] you date But term you have injection and give you have initial Rex initial Rex is d cold start date they see every so this is sort of an landscape a very over simplified landscape in you dimension and a given this dimension is popularity in this is d items life it looks some new you old and a not so hot so when when you are cold starting an item it is new and it is not popular so date mens it means you collect some samples target d right users and thoughts ho they no weather it is going you do well and not and if it is doing well it gets an head break because it is popular and it is trending a but if it is not it just devise in something date can be pic d oil ricky so ho they tomorrow if I tell something natural come in you picture an where an you you match you user and give them recommendation date is totally only dare totally something only they but d think this If you don't do other content based filtering what will happen is collaborate you filtering typically two videos have item based on library sorry they have item base collaborator but they style take that and signals always so what will happen if you don't and if you don't have a method date country note that that and signals a everything bill convert you it's headark so finally at some point of time you bill see you bill collaborate filtering bill be serving just head recommendation and this bill just growth in your thinner and finally you bill I mean totally depends on your top of d fan so date is an bed place you be in so a di really it's that they need content based recognition and um d other think is an time starters converting you an particular that and profile it's almost like you know what Steve Jobs right you don't know what you want that's pretty match further but serendipity i mean they try to give you something dat they think you really like so you didn't expected this her idea give little literature youtube typically see the third of the sentence is recommended i.e. expected but useful cabbage on relevance and look but interesting this very philosophical am but d actual point it helps avoid ahlabation in d sense it basically expands whatever but also loses exploration it needs u discovery which is d more d region wahi pillar apan sold have it because they are a personalized discovery engine thoughts what they yesterday sir soya aa but d challenge is n ho tu survey good content dat is not random is n expected but useful right and be two i major and control of this but one of d things dat they did something close to u this so second pti you give you but i think they did this and this is what n I want Dumbbell one you do so basically it will be awesome if you can do what ever you do and Wikipedia on the entire and you know on internet so content understand d basic information retrieval based NLP stuff they are not they don't do per since always come d statistics machine topic keywords language us content typing missense youtube would be n video number of ads number of links responsive design like its important feature per s development they have devices overlays papa and let's mark in this talk what are concerned about an topic basically topic modeling because that's where I am where they are going you discover topic date they want you you make an you recommend and peter content and or you going you every d word indoptv lot in this stocking model what are they basically language model basically help you understand d you know underlying structure of document and semantic understanding of Content so date you can index and make it on this entry application is basically continue which bill do next light a dimensionality reduction and modeling this is applications content categorization so they want to categorized document so this has some of the retirement date they have in their up what they are keeping this stable has a fixed tax on me at d moment and d so they want to be able to categorized document to pre different topic on me so date they are able to handle lots of lots of things like per example user categorized something they want to evaluate whether d user categories it correctly may search in recommendation definitely a and discovering related interest they want to recommend make your profile as rich as possible so date they can survey but recommendation so these are direct they are attacking d user n group this is Where the idea recommendation are going you come you so they get a feature extraction basically at every point of time they are going you choose what you pass and based on the content type and if it is text they have in you pass it in one particular way but if it is an image video you are going you pass back and they are based on domain choose so you get read of boiler plate so some and some domains you know traditional boiler plate plotting library don't know so they have customized version plotting code detector language choose and based on the language they thing what analysis strategies you know you proceed with strategies and and grams ming strategies and which data clean up remove stopwatch stem and they don't always do this steaming part and and and cleaning cleaning part it depends on it work on the specific topic model but and grams they cross verify with Wikipedia basically thoughts and thoughts and Wikipedia graph the article graph Basically they have it on wiki wikipedia minor and they categorized it by the features that they have on computer and you set categorized on categorizing and first think that they do because they are discovering topic directly exact think that they have in dumb ups and like this classified is very simple generated from one mixer so you are going to get it you go an nice products on d number of features that you have you are going to compute this time good some nice properties and some limitations it has so it is supervised and generative it is efficient in both training and classification I told classification it is linear it is easy to implement an online version now they can and they have an brush and a human which it is an online processing library and something dependent on static vocabulary so this is elimination right because vocabulary which till if they want to change Vocabulary they would need you to read train it and they need you to know do it do mean adoption something like dat aa and d second an ferris again coming back you d idea of this you d idea of this selective random so called randomized aa serenity this in cute cut it because restricted means of identifying semantic group thoughts d thoughts d property of mixer model aa where they have you think dat which words if in an mixer in and distributed in 1 particular way you never think dat d distribution can be different and can be different per document and so on so thoughts same d next obviously think dat they co par su you latent this allocation this gives so more room to move around basically so latent allocation simple it uses just you parameter and sometimes if you have an topic word distribution constant give young day it wikham just alpha alone So here basically ho ho d distribution over you know ho d topic document distribution are going to be its parameter per discipline distribution which in turn generate from topic this is ho d topic distribution lose right topic document distribution index so wherever higher alpha is n it's going to be much flatter and d lower alpha today it's going to have it's going to be parts so what they are saying is by selecting lower alpha and setting that document on and average is going to have you have only one note on topic and you and three topics where when you have very high alpha what you are saying is it can have other combinations of which d topic d se think goes per inside perimeter date is missing every eta is again so again in this don't know its distribution over distribution basically it's every point a date sample from edition distribution is Going to sum up one it's going to be every probability distribution so date this date is what is really called about it so you can think of an topic matrix so you would basically have an on every topic you would basically have the word probability and which of date product sum up one right this is them generator you know join probability basically probability aa coming back to you d interesting properties supervised and generator I can make it supervised by putting an regression model together with NDA it is very well sited per dominate adoption so date mains I can an 10 point of I don't need to worry about tax animation I can change d tax on me and I can an I can pretty match run d from I bill go dem a get in this in this is values per go dem a get in this in this is values per doing recognitions or this is d think dat I was talking about this is very helpful per san I bill come to you it some time can be Extended you discovered relationship here is objection so basically it gives a lot of room to adopt topic modeling in other direction date they want you and date is really call so how do you evaluate an old model is typically in them publicity which is some a just a different scaling of a you no entropy it is an in something it is only it is checking out an house surprised this particular ups d model is basically how do you surprise this model is having seen this new test sample so set of test samples so they also use human judgment from n word impression and topic impression test where they say date n pick out d word n pick out d topic dat pick out d word n pick out d topic dat ten and co well with d ideal listed here you can also saying son can me not bird n it can be static it can you can if you have a fixed tax on me you can always n take d patient model mixer and initialis a NDA this This set up mixer going to d next light but coming back to this this this is sample set of mixer very very good qualities really good know what you need you know this this is how things look when you an him d right I think d d recommended beta value is 0.1 and d alphabets 50 / number of topic dat is d typical think dat is 10 on scientific papers academy papers but they want things to be a little more loose right so thoughts where you can get you play with alpha and beta you can increase beta and you can an reduce I mean increase also a little bit so dat dewar is note note variance an in d mixer they give about doing d next think they tweet it say an tweet d recent idea of using dimensional altitude and ldecording model using d URL in d head and n they give him d model you classified tensors m in d latent topic space similarity between documents with topic overlap you can you see similarity your weight is tight but you can also you know this topic and rap dem in some sort of licht aa and computer livingston distance and humming distance d call think about date is you would be able to interpret you can cup you can say date i don't want don't put document it is a very similar in d graph aa just a at least have one at least have to change in d in d l ha so date i have loosely similar document but note very similar document in d graph wher document make up d notes and d similarity score make up the ad switch give way r page run way run personalized page ranker in this how it is an the other name this topic sensitive page run with d document craft date allow is you spot influential document on topic and index on faster dream the idea simple started random note and you an only this gray notes r Basically now the actually good amount of you know chase you be widgetized an advise this is d transaction probability dat is moving from a you b and b you c and things like dat and you move connect note with d probability of 1 - alpha and you move you something dat this not connect and you just teleport random with d probability of alpha so dat way over fu iterations oo converts their stationary distribution d whole idea of topic distribution d whole idea of topic sensitive verb personalized page rankers you m every per example we n initialis a which d know its with and 1 / n probability you say dat if this think deing belongs to you this particular topic make n zero an if it belongs to you this a you make it n membership matrix right you would basically run this so i m like ho i have n topic you have d data mins and you would get different page ranks on different topic no you can Paralyze which does and simultaneously and impact you have inside go tomorrow d fast personal life ranking personalized page rankers vipr time date with just yes so basically it's with this you bill have you would be able to index ds contents that are very influential in particular topic whether loosely connect and and they have tell rik so does are also controlled they jump and they have some set of control in d sense date if it is they you lose we know how to tighten it by controlling d parameters m and how do they value it now testing me really me be testing a basically recommended item versus a random pick item ho ho is ho different is d user behavior is c date a serendipity like free stumbling sessions session is basically group of recommendations so they recommend sessions also right and something because they cash things and so they look at serendipity free stumbling sessions and Sessions with just an individual recommendation and personalized recognition and put them together and they see the difference and look at implicit x place signals and retention ongoing and future is basically more online computations where aggressively looking to computer everything online data max data cup learning think very easy they don't need to worry about it which model feature improvements basically more stuff they already use some of this but they want to use it in the best way possible and added dictation improvements again do do action so its an to problem and on different cases image video text you need to do different things on news which d things are totally different but recommendation attack its different reaction group of Wikipedia mines Wikipedia mines stall k thoughts an I think people from sentence you university new d people be hand vehicle I am not sir they come Up on this and it's really called it take Wikipedia doctor and there's a lot of classifieds and also you to unity document using articles and it has it on it looks like the prior probability of linking taking a surface from bird and what is it the probability date date surface form would be link to one particular article and it know you d some big weight no date so door me b change yourself like a paper doing paralyze similarity collaborate filtering it just using d some features so right now they have tread it but they want to experiment more with it and they don't have a model date I know what you are talking about it's a joint model it's overall generative model they would love to do date so it's one step with d time so this typical lot of father features date comment you play like and if your serving content based they always in d taking usage signals so the results are always Mixed you need both you can't with either but thought it and this is one possible way in which you can just content features alone you rank step it like google did so on thinking based signals which i they basic communicate which d matrix bikers again a dees are some short term trick date can be onyx and d whole like it they want people to dumb more and show certain n you know they are looking at good things with d function of time and r sun like spending and jo dat but it's devar is lot more tu it devar is n big they have an implicit scoring model a date device lux at you today n him ho match time the question I have is more related tu aa is youtube with d LDA model video evaluation n real time out after search and you have bin in give you married in Sanskrit model d evaluation is going tu an offers d human based evaluation girls and before b settle on an initial model right aa but after dat bill b n evaluation constantly devar bill b n polling constantly have happening so they bill b looking at d publicity and once it goes below particulars significant m when it riches 1 particular platform and anger sorry goes up they decide after you can every this n user get tu search ok i see so recommendation is n problem of sach aate d back right just Date you it is in query week you create d query set double problem first you need to solve d search problem but you also need to solve d querying problem so you need to know okay you are d user what you do 24 so desert answer yes I think 500 plus now this regarding your statue craft don't have they don't have topic sensitive page ranker okay so they are doing everything on a single machine but they are they are coming up with distributed tha I have don't like it I think