Chris Moody Q&A with Alexy Khrabrov on lda2vec
Recording: Chris Moody Q&A with Alexy Khrabrov on lda2vec
hello everybody I'm Alexa Cara bro the organizer of sf's Collins a spark and he'll have a guest knit up we have san francisco bay area machine learning which is our sister mahtab hosted at nitro tonight and it's also a joint meet up with SF text and here with us who have chris moody who is a data scientist at stitch fix thank you very much for having me Alexi it's great to have you and I must notice that Chris gave a talk last year text by the bay which is one of the highest rated talks which talked about work back that's right I and many people remark that they finally understood quarterback and it was really great so today we have a new algorithm called el dato back and looking forward to to that but no before that and what asked increase stitch fig is very interesting company we have multiple talks in data by the bay in may from it can you describe the problem and how it leads to so many interesting data science questions yeah absolutely i I mean absolutely like stitch fix is an unbelievably interesting problem and I think on the when you first scratch the surface you don't really you don't really think it's super interesting right so for example you know two years ago I was doing a physics PhD doing astrophysics and supercomputing and if you ask me now if I would be doing clothing especially like women's clothing and let's get out of here right but once you start to dive in a little bit on the surface you start to see all kinds of really interesting patterns for example like fashion is like this this this incredibly interesting conversation that a whole whole community of people always having with themselves and it changes over time right so trends are changing in time and they're extremely like sort of sparse they're very high-dimensional and they're changing and one of the ways that you can actually try to understand that right so for example you know leather might have been in like last year like tiny little other straps but this year it's like patchwork denim right or maybe it's just like and maybe that's like a full month trend maybe it's like a full weeks trend and so if you want to capture one's individual unique style some of that's going to be made up by this sort of like globally changing component some of its going to be regional some of that's going to be time so I mean it's going to be individual to that client right and the way that we try to understand that fashion in those trends is by it the same way that people describe that clothing mm-hmm so that's why text is such a powerful useful thing here because otherwise it's extremely difficult to find these trends you have thousand tens of thousands of items you have like this really sparse time series changing all the time and that's why it's such a thing such an interesting problem like how do you solve it how do you predict all those trends right it's basically a stock market on fashion mm-hmm and at the same time you don't get like these little stock tickers out you get like textile right mmm so and you don't get just the text that you get it for individuals loading onto like those individual things so anyway I could go on and on Oh interesting so your life so basically so the users describe the preferences in text how large of these texts are the Twitter side the paragraph sighs how long the descriptions totally depends and so it depends on like where where you're talking about so right at you know stitch fix I should actually probably explain mm-hmm if you go to the website you see almost nothing mmm have you go you register you tell us a little bit about yourself what your size is what your weight your height and like we ask you a few questions about do you like this shirt or do you like these jeans no yes no mm-hmm very minimal registration and maybe you fill out like a paragraph about yourself that maybe like tells us something like like maybe like you're an expecting mother mm-hmm or maybe you know your your your work in medicine in some way and you'd like clothes that aren't just scrubs right all right and so that's like what we have to start off with and that's what a stylist then looks at tries to send you like the right bit of clothes and you don't get to see this close before they show up and so that's part of that's part of the promise right like that will try to find clothes perfectly curated for you mm-hmm the box shows up you open it you put on some of these clothes and you immediately start to like you know maybe you hate it maybe you love the fit or maybe you love the color but something else was wrong or maybe it maybe we knocked out of the heart and got it completed right by the way you write that reaction down right and it's really really hard to design a survey that just gets every single possible thing that you could have like you know maybe it's like the polka dots like that we're kind of strange and they were in last year but not in all right or maybe this particular kind of fabric like you're allergic to right so those are very specific concrete xbox for feedback which can be free-flowing text sure mm-hmm but the big challenge here is that how you describe style is you know it's an artistic concept right it's really hard to build giant sets of attributes over this but the text get you that mm-hmm and so that's why we're so incredibly interested in all of us because we can actually think we can build a style space we can think that we can think that these clothing items are very similar that these clients are very similar because they describe things in the same way and turns out to be a very powerful idea mmm so you see that the kind of bears all that people can be Gloucester's the styles can be identified so the fashion is then like the collective trends in the space interesting so you probably are like one of the few people who have an idea you can probably see like computational define fashion that's right and I think that's a really really really motivating thing right like when you look at like the trends and machine learning like and like there's lots of barriers that have fallen the last few years right like alpha go one like a few few months ago mm-hmm word davec when it first came out you started to really feel like you could understand words with all this with a lot of like deep learning on images you start to understand that think that computers can actually understand images I think that we are very close to understanding fashion right and fashions a very art but it's also personalize mm-hmm and I think we're just at the edge of trying to describe those things quantitatively interesting do you see kind of do try to define a distinguished fashion by geographical area by country like do you see kind of it gah Stewart in that space I think that's a really interesting question right like how much of your personal taste is say influenced by your friends right how much of it is influenced by your occupation how much region how much of it is unique to you right like how much of it is a reaction to who you were a year ago right now for example I go back and look at my closet I look at the clothes that I had like two years ago I'm like I'm not wearing that anymore right I'm perfectly fine clothing but it's it's out of fashion right alright so and so you know you coupling to yourself is also like an interesting sort of like reaction I can happen right so now what how do you you can write that down as an equation how do you find the coefficients there how do you find the local features how do you find what's important I think that's like what our algorithms team at stitch fix's trying to do in a large part trying to describe our clients try to describe our merchandise and tryna style or trying to figure out what the whole styling experience is like mm-hmm so I understand the guys are involving human in the loop oh yeah right so how he could talk a little bit about that because it's kind of combination of humans and machines yeah that's a huge part of our business and I think for us anyway it's an enormous competitive advantage you know not met the many I think it's easy to become obsessed with writing the most awesome machine learning the most awesome deep learning algorithm ever but I think fundamentally where we're going to be for at least the next like 20 years is seeing new forms of like humans interacting with machines and so that's where our H come to our human computation team comes in handy right so we have a we have several thousands of like a fashion experts our stylists and that is a computing resource right that is that is that is a pool of experts that we can ask and answer questions from like what does this style mean to you yeah thanks to this tile can you try to try to interpret this style in the way that our computers can do something with that data mm-hmm and that is incredibly powerful right and so suddenly you know even like at a very low level we get things like you know at the very least we get like labels for our data sets right and it's certainly start with that but i think there's also really interesting parts like what is a machine really good at and what is a human really good at 41 the human is extremely good at forming relationships with our clients are stylus extremely good at forming relationships understanding our clients of capturing trends of capturing all kinds of different context of reading the text for example of looking at their Pinterest boards and try to figure out a cohesive form of their style what's the computer really good at okay it's really good at wrote calculations so like a lot of things like this particular item is extremely popular like we should you know probably put that up there mm-hmm we should it's probably really good at figuring out to start the sizes of different things lots of things that can be like numerically quantified like don't make don't make a stylus do an inner product don't make her do a dot product in her head between client features and item features right right but do give the information that a stylist can then use so mm-hmm text i think is actually a very big example text is somewhere where a computer can kind of understand a little bit about what's going on so like you can kind of do searching like searching is ok but understanding the nuance whenever she says that you know i want exactly the same thing all right you know i love this everything about it but uh you know i didn't buy it because it's half a size too small mm-hmm computers is going to just see that she didn't return that item it's going to be a little bit hard to parse out exactly the right kind of action they should take their that's exactly a kind of thing that you can then maybe surface and bubble up to a stylus in a particular way and then a stylus cannon like action off of it so it's this interaction i think between humans and machines it's like super super interesting and it's and it's at the heart of what our human computation team is trying to do so I guys try and basically to see where this device should be what kind of actions you should propagate the stylist you can see what they're going to do yes interest absolute and that's a really really interesting boundary really interesting because you're seeing it change like every year every year like new algorithms come out and and there's a new way where you can interact with stylus like you can make stylus more efficient like you you can use tools and new ways and I and I think it's largely driven by things like computers can start to understand images so maybe maybe she doesn't have to look at the whole maybe the client or the stylet doesn't have to look at like the whole Pinterest board there's a hundreds and hundreds of pins try figure that out maybe the stylist doesn't have to look at like every single comment that is that a client has written maybe we can show just the most relevant ones still let the stylish read them a notion of what's going on and action off of that but showing just only little bits that the computer has deemed relevant in some way and so that combination of human and machine is only going to get closer and closer and closer I think with time and now what I what I think would be really interesting is if other companies started started having sort of like human computation teams inside of their own instead of their own infrastructures it looks like it's becoming human in the loop either the last year we had a few companies with a data lingo had stitch fix and since then i think i've seen more kind of companies appear right i think the term kind of become more widespread human loop so i think it's really interesting and you guys obviously kind of have a lot of work so here's it's just occurred to me when I describe this I have an interesting challenge for guys all right or kind of an experiment right so as you know data by the vac on responses galvanized yeah we use also a cork in space home for many startups and obvious lot of startups around by young founders so every time I can't there I kind of sense that there is an image right there is a fashion for the start of founder San Francisco right like there are lots certain kind of jeans are kind of shoes so kind of bags and jackets and obviously you have to fit that image let's say to pitch the VC right doc signaling in a certain way the URIs are the founder right so I'm curious right and so do you guys skater two men let's say I want to I want to dick out myself as a solo founder let's say I can make a lot of pictures like few pictures you know have a lot of funeral pictures from a conferences where we have a bunch of kind of you know galvanized folks and genuinely sorry follows right so let's say you know how would I go about you know and like I just arrived from foreign country like the Russians completely different I kind of sense you know like in order to make it here and recipes on the founder but I cannot really put my finger in like any basically personal tailor in the whole times right but like obviously I don't have one so I'm going to stitch fix yep cover the go bother I feel like you've just given my whole pitch for first dish fix men's we launched dish fix men's uh we think last week all right I don't know that so is correct it's currently in beta say if you want an invite and it sounds like you do yeah I I was threaten this problem Everett I'll come to going on but I mean that's you know that's not a that's not a unique problem right like people need particularly close for particular occasions mm-hmm and if you want to be successful you'll do that right like especially in this case right right if you're pitching to some VCS you might not know it you know like look matters right right like it's extremely important so have a stylist figure that out for you right it's like a nice thing to have yeah that's right it was this joke in the 90s that you know in New York that if you want to hire a really good quant you go to JFK Airport you find a guy in a really baggy suit in like refreshing glasses he's probably Russian mathematician visits you just fire them and like they'll be the best one right right so because you know in Russia mathematicians business didn't really care about goals and in the 80s and 90s did have a way to care about it that sounds like every mathematician I've heard of yeah so so wildly stereo yeah so you glitter look at Sarah time right and another Joe comes to mind some somebody uh somebody made a joke around Halloween like you know it's kind of it's ridiculous how adults are dressing up in little suits and kind of an and then somebody reply to this well every day you you put on a dress suit and look like a big guy in the big company like how is this different right so it's kinda but it was interesting obviously there are contexts right so and that's one thing that a stylist is extremely good at figuring out as compared to a machine mm-hmm right you know your your your case was very particular right like you didn't go to just like you didn't just go to work right like you didn't just go to interview you went to go interview with a VC right right and so like that means that like whatever might have flown for an interview attire in San Francisco is now stepped up a notch mm-hmm and so that's the kind of context that a human can read and try and figure out how does the human communicate that back to the Shane right that's an interesting intersection mm-hmm how do you do that how does the machine except those inputs how does the machine talk back to the stylist and given like new recommendations condition on whatever this year's just told her right so there's really really interesting like cross-feed a you know like you cannot really run deep learning on this because you will not have three million people doing vc interviews right like you'll have barrel a few cases I mean you will have an image right like basically like we know how to stereotype right that's what but you know how do you explain to the machine like you will not feel it a lot of data yeah I think I one of the things that beepin is really good at is having supervised classified data sense right and it's starting to verge more tour it's like unsupervised work like el dedo vac is going to be talking a lot about unsupervised learning mm-hmm and we might make inroads there but for the time being I think that the biggest gains are to be had with how do you optimize like machines talking to humans and humans talking back to machines and I think that's that requires a very specific set of algorithms you need algorithms that don't give you sort of black box results like how deep learning usually does right which is unexplainable which don't get me wrong I'm actually really a big fan but like usually you know there's two different kinds of explain one is like do I know what the model is doing and then to given those results can I actually understand what those results me so this things are different one I think for deep lending in which it sort of feels on both of those things like you have a bunch of layers in between you can kind of visualize what those layers are doing you can kind of visualize them for like individual images and get a good sense but the output is this rather one of those intermediate layers is this sort of like big giant four thousand dimensional image vector and you don't you don't really know what that right you look at one of like you look at you all you really know is it's similar to these other things you're like okay you can start to figure out stuff from there on the other hand sort of on the other spectrum of like models you can do are things like graphical models things like lda which by their construction which are from the get-go not trying to be the most powerful algorithm in the world I'm going to be the most interpretive algorithm yes and the results that it gets you are extremely nice to read off right and that's the kind of thing that will let you go back and say here's the my machine our learning algorithm here's what it feeds back to the stylist right I can't give back a word to vet vector back to my stylist that's not going to fly I can kind of try to give her an Lda vector at least that's only going to be only only essentially because it's only going to be three or four numbers that's quite a word to backpacker which is going to be a hundred dollars i'll explain a little bit interested later today but like that family of algorithms is what you need what we need to push forward to have better human and machine interactions mm-hmm all those interesting I think kind of the the the meetup audience senses this because we have a record number of enough today we have 300 people all right look at all these girls who want to see it so obviously i think board tyvek was a really huge hit with the community right and and but under a question so i think it's like you're real really are you sure it's not just the machines getting smarter and overloading the meetup I think there are other situations they're gonna actually know this time to think about it you know it's actually is interesting because you know the weight meetup works right when you start a meetup group right now sorry Sarah meetup groups you actually you provide this very similar you you know it first you give it a free text description what this meetup is gonna be about and then it matches that give you some keyboards back and then you have to select right for us you know let's say I can you know certain sparks I will say this is a bald and I remember computing you know this is about kind of type of program with big data and so forth then it will suddenly come back and say machine learning data mining and then if I select these keywords I think it does proximity affiliate and it's finds new keywords and I think for every user it basically has a similar system so i think they have somewhat similar problem the problem is that they don't have a stylist so you often get misclassified but i think it's actually very good at Wrestle so we actually I think get pretty kind of cohesive audience so I'm very excited like they really kind of dig this topic yeah you know what I'm interested in I know you said that it would be interesting right what if you did have a human in the loop there for meetup it's probably one of the scales yeah I'm done it before but but let's think about it like other domains right like like alpha go winning last week mm-hmm what would happen if you gave alpha alpha go recommendations to like that top notch player yes Liesl right uh-huh would he perform a whole lot better how do you actually optimize that like what do you think would happen like I'm really interested in that right like hey what if we had 10,000 GPU machines at his like at his uh at his command right right how like how big would like what if you had the hundreth best player playing against lee sedol and then had alpha go on his side interesting ok what's like how much machine and how much like human can and how well can they work together in like things like games right would be super cool to see right I want to know where that would work out like interesting because because it's maybe it's a different skill all right because you can be a great go player but maybe you can be a great commander of go play machines right now wouldn't that be interesting right like wouldn't that be super interesting like like would it be great to like kind of like say to your go algun like look I kind of want this like level of strategy like like give me back some percentages that that's going to win like you know that's what's something that the computer is going to be really good at brute force like stre searching across a bunch of different things but maybe that the human is really good at like sort of like sensing maybe the long-range like strategy or maybe some of the short-range ones right like figuring out what the right mixture of like the global thing is that would be super cool to see right that's how I think of like to bring it back to stitch fix that's kind of what i think of our stylist is doing curating the sort of like long-term client experience of like what are like the individual items like what's for style learning about that and sometimes you don't send items you don't necessarily send items that they'll immediately like her that you're sure that you'll like will send items that explore and push them in different directions right and so you might not like it you can send it back but you're telling the style is something with that and a stylist is learning something about that right and so there's an interesting like push and pull here and it would be really interesting if just like alphago player a stylist could explore lots of paths going forward and see which paths are the most interesting and then answer this sort of like explore and like and hate the word explore exploit but this is sort of like what the field is room it seems like you need to really cultivate your best stylist thread on see how they do so I mean obviously I'm very intrigued now I'm gonna sign up with a tissue is better we should just learn in this conversation every little girl honest to god right so but I mean actually you just give it back my hope of the human race because I was very much said right I was always thinking that if I do something wrong right because of a circular like harness and gigantic machines to squash little humans in things like go obviously like we're still much better and many other things but it's kind of its kind of gives me hope that you know as long as we're in charge right like now we it will be obviously humilated machine aided humans right like in that that can that will come in many many contexts is a lot to be said about this interaction between the two and i think it's unbelievably interesting on that great notes I think you're a much and we're looking forward to your talk some thank you much