Text By the Bay 2015: Lyle Ungar, Measuring Well-Being Using Social Media
Recording: Text By the Bay 2015: Lyle Ungar, Measuring Well-Being Using Social Media
foreign ER I have a somewhat different take I want to give you on understanding text and social media I'm coming from the world well-being project which is a group of psychologists and computer scientists trying to look at language not just as we've seen mostly here which is pulling out facts entities relationships not just which company IPO would win but looking at language as indicating things about people telling you their personality telling you their moods telling you how they're feeling it's a rather different take on language and one that I think is quite good in terms of actually measuring the well-being of people how happy they are how healthy they are I'm going to try and convince you that first of all those things are deeply connected people who are happy and good relationships live about five years longer on average to other people that's comparable to not smoking so you can actually be happy and have good friends you will on average live a lot longer and I want to argue that the good way to measure that is not with your Fitbit but with your phone that the language that you're sending and receiving tells a lot about you and so today I'm going to give you a whirlwind tour of a bunch of results we have if you go to wwbp.org but I'll email that if you want and see lots of results that say what can we tell about people from the language they use I think it's important to realize that language is not Inspire what one thinks maybe here in Silicon Valley mostly about explaining how to run your Scala code language is mostly people sitting around talking to each other chatting to each other telling stories about how the party was last night and who they liked and who they hated and why the expletive person lots of expletives sorry I'll show some of them did that so I want to look at Facebook and Twitter and try and look at this conversation to understand people their happiness and eventually their physical well-being their health make sense I don't need to tell people here there's lots of such data duh okay so we can process it what I do want to contrast that is how traditional psychologists work they take 50 undergrads give them a survey ask a bunch of questions and try and build a model big polling organizations like Gallup have done a million surveys okay I'm not that impressed nor are you right cool okay but our stuff is fairly big for this group it's it's nothing so what but compared to most of psychology we're by far the biggest collection of psychology data people that I know of so it's very big I'm going to tell you a little tiny bit about the instrumentation we built mostly to give a flavor and some of the contrast that with conventional psychology then I'm going to go into two case studies what an individual level talking about people's personality from the Facebook and then we'll apply the same sort of techniques to looking at communities we can build personality profiles from communities that predict their happiness and how long people will live so historically psychologists have built lots of dictionaries positive negative words negative words you're all familiar with the problems of ambiguity historically marketing and many people here have built Cinema analysis you can't see this but people find positive words and negative words associate them with products and new product reviews I'm less interested today about sentiment analysis do you like the product and more interested in a different sentiment how do you feel how do you feel yourself how do your friends make you feel right so not just thumbs up thumb down product but looking inside of people psychologists have spent lots of effort trying to build up dictionaries that measure things like positive emotion engagements relationships meaning accomplishment you'll not be surprised that take ten thousand tweets M took them build a model you get much better models than five years of clever psychologists writing down lists of words much easier to figure out from statistics what things correlate with a feeling of accomplishment than to try and guess accomplishment words and you get all sorts of funny things when people try to write them down automatically so what do we like I like a lot not just building up models we have lots of models of all these things I mentioned but also looking at more data-driven word driven and one thing I've not seen mentioned much here in spite of all the words of X is latent duration location people know LDA if you've got so many don't so wonderful technique nice open source software take a bunch of documents right say 100 million tweets and cluster together words and find words that show up in the same cluster words that tend to co-occur and you get groups like this you can sort of see days of the week plus a few other slightly garbagey things plus mornings and nights and we build typically a thousand or two thousand of these here's another one things that you might not have thought of grouping together but at retrospective okay these are sort of similar if you prefer you could do a first step and go a word to VEC or eigen word or some sort of vector modeling cluster those but the notion is that you build up say 2 000 groups of words I'm going to show you a bunch of these and then you find for each of those two thousand groups of Words which one correlate with the outcome you care about like extroversion or youth or maybe even buying a product so we then have a nice infrastructure we grab from a bunch of people either Twitter or Facebook we pull a bunch of features engrams multi-ord expressions or phrases these topics these LDA clusters called topics and a bunch of other prefab predictors we correlate them together against people's labels they tell us are they male or female how old are they they take questionnaires and we come up with some sort of a visualization and I think whoops it's pieces there the main one again I want to come back to this notion for a lot of data there's not that many pieces we get maybe 10 000 maybe 70 000 volunteers to give their information and if you want to regress that against 100 000 words it gets sort of sparse so very nice to use these sort of topics and you get all sorts of you know pain blood Hospital teeth or family topics whatever a nice way of sort of generalizing over various words cool so let's go out from that and stop instruments and actually do some results fun part so first I'm going to talk about individual level results and then I'll talk about Community results so at the individual level we have 70 000 volunteers they have opted in to share all their Facebook posts they have told us whether they're male or female how old they are and have taken the standard personality questionnaire now in Industry everybody knows the Myers-Briggs but it's out of fashion psychology is not being well validated there's a similar sort of questions that's widely used called the ocean model or the five Factor model or the big five openness conscientiousness extroversion Myers-Briggs agreeables neuroticism people take a whole bunch of questions and then you correlate their outcome on that question which the rating of Personality with their words make sense but before I go to personalities look at things we understand better which is say male and female so first of all all consent let people know you're collecting their data they have to opt in to share Okay so what do girls look like on the web on Facebook okay so what are we seeing the the actual bunch of these pictures the size is the degree of correlation how much these predict femaleness versus maleness and the colors the frequency so the heart and the excited are both very frequent and highly correlated saying like it on or wishes she by the way I'm not a big fan of single tokens English is massively ambiguous why is this Chinese or any language I know about um but if you do a reasonable Mutual information you get these nice little phrases wishes free she loves her I'm so happy which tends to solve a lot of the ambiguity problems really cheaply really great hack oh and of course what do I consider a word anything people use guys don't use four O's very often if you're so excited it's a girl thing yeah okay um what else am I here boyfriend you'll see typical thing that girls talk about okay before we get upset what do boys talk about ah [ __ ] I've got that ouch really embarrassing now I got to point out the vast majority of the language is the same words like the it the same between men and women but these are the words that are most distinguishing of males in Facebook small sample of 70 000 people but they're all statistically valid it's not all embarrassing there's government and Engineering um the World Cup Sports shaving some things are more subtle Xbox gaming is a very male thing yeah um note my girlfriend and my wife it turns out we go back and look at the original Facebook posts women talk both about their boyfriend and their husband but off but other people's boyfriends and husbands guys mostly only talk about my girlfriend not about your girlfriend again pointing out this utility of not looking at single words which wouldn't show that but noting that this multi-word expression is in fact highly significant make sense yeah okay so that's male and female how accurate is it we can take Facebook Words gender 92 accuracy telling whether you're male or female from your Facebook so it's not super good but good enough to do statistics on and certainly useful we also have more interestingly personality the big five what do extroverts people who fill out rate highly on the on the extrovert things these are controlled for age and sex they're kind of fun you know can't wait Amazing by the way to State the obvious again notion of words what's interesting about this can't wait it's got two things it's about extroverts how's it spelled extroverts tend to drop out the apostrophes right so again if you're trying to capture personality spelling variants are highly significant um probably find other getting right more frequent in the South but also more frequent with extroverts don't again see the spelling undone love and bout feel it extroverts are dropping those G's they're dropping the apostrophes they're getting out there and doing that they've got girls and somewhere there should be boys and baby and love you well this is great except unfortunately most computer scientists are on the other Spectrum side um so what do we look like uh well on the internet computer so I I didn't make this up don't want to you see things like drawing and books reading um it's not all bad there's things like um depression yeah that one's not so good apparently words of of more cognitive complexity thinking a little bit of seeing both sides related um yeah it's sort of funny I'm not sure I'm really happy being slightly on the extrovert side um but they're good size let's look at different pair I've got five of these I won't show all of them neuroticism what is it like to be neurotic um really bad right so you see the squaring you see again note the disambiguation they don't talk more about sick but they talk more about sick of right and again these are not manually curated it's all automatically generated by grams and trigrams by just visual information models um yeah I hate depressed lonely scared my head screaming okay this is not a surprise to my psychologist collaborators but what was a surprise is the next slide which is people who are extremely low in neuroticism sometimes called well-adjusted people what a well-adjusted non-neurotic Americans talk about that was a surprise well I think less the weather the note it's beautiful day so it's a positivity thing there the second minor one besides lots of sports is interesting because it's both you know Lakers and mascara mall but also snowboarding and the gym that is religion and the religion is not so surprising on average in America although not in Europe religious people are happier have more strong connections more people to show up if they're sick and go in the hospital so in the U.S religion is a very strong sense of community and correlates with well-adjustedness but the sports is a novel one to the psychological literature sort of interesting is not just life is good but um and things the shop well I don't have time to look at it but things like mountains it turns outdoor exercise shows up if you look more subtly in these so it's nice you get a picture of what it's like to be well ingested do more Sports golf uh no but these are these are these are remember that these are are a sample from Facebook golf is not nearly as widely talked about as basketball or laughs could could well be so we've got a bunch of these we can have agreeable people disagreeable people open people doing music and writing disagree um closed people not open people again can't wait and don't you're what conscientious people so you want to hire the ones the agreeable ones and the conscientious ones are the ones you want to do when you're hiring that's the people you want to work with content disagreeable ready for the conscious people talk both about work and great days but also you look more closely about ready for dinner and weekends and eating out the non-conscientious people well again swearing and Pokemon and yeah I don't know it's all data I you know I just report the statistics these are all everything here statistically significant because the sample size is big enough um is there a causal connection was it I don't know but there certainly are difference differences right cool so how well do we do well here is the correlation how accurately we can correlate our predictions with the outcome of the exams these testing for openness conscious extroversion growth and autism our stuff is in blue higher is more accurate in red are the ratings by people Facebook friends and what you see is for most things were indistinguishable the blue and the red are indistinguishable it was accurate as people's friends are at reading their personality except for openness where we're actually much better I think partly because it many of these constructs what does it mean to be open is a little funky and there's a couple of different sub-facets of openness there's one that's that's more open experiences whether it's more sort of poetic in writing so I think maybe the friends didn't quite understand the definition of what openness was as well as the computer did but the fact of the matter is that we can tell roughly as well from people's Facebook posts what their personality is as measured by a well-validated there's easily a thousand papers published it's a whole Journal of Social personality with all these results um always small sample size compared ours so we can tell personality as well as as people's friends can which is to say it's you know noisy correlations are you know 0.3.4 but lots of signal cool it's lots of fun um how does this put here's the same well-adjusted Sports One I showed before at an individual level right this is 70 000 individuals taking our survey we've also done a bunch of work at the county level which I'll show you in about five minutes taking tweets mapping them to us County measuring the county-wide outcomes and correlating those with language and interestingly counties with less suicide in them tend to also tweet more about language the same way that individuals who are well adjusted and non-nerotic tend to talk more about sports in Facebook so there's an interesting connection between individual level modeling and Community level modeling that many of these things span both levels we'll come back to that later before we do that let's digress a little bit and just talk about age just again get a little sort of flavor of some what these things look like and I'll show you a sequence of plus we have you know the age of all these Facebook the 70 000 people what does it look like to be a teenager and the center I will show you the same word clouds I've gone so homework so they talk a lot about haha and if emoticons I don't even know because I'm not a teenager and school tomorrow and then around I've shown the the topics again we took two thousand clusters of co-occurring words grouping all the words on Facebook and these are the ones that most correlate there's a lot more so what does it look like to be a teenager sort of cute you hug and you date you got school tomorrow when you get upset about things it's you know OMFG yeah FML you guys know these things you guys are you too old you're too old Gabor is looking at the same I don't know gah okay let's move a little closer to your demographic um what happens when you go to college semester you start swearing which you didn't before um what else happens no sleep anymore working hours teacher and I think I'm feeling what happens when you get into your whoa whoa whoa this is evil get away go away I don't know my computer has been possessed by demons that's another story um the crackdat is clicking randomly by itself what happens in the 20s this is a lot of you you are at work and getting drunk and you're taking a quick nap you're getting married you're having dinner some of you are cooking at home not you guys but other people are hunting for jobs work what happens when you get in your 30s and 40s my son my kids my daughter family and friends lots of friends oh and for the first time religion is talked about who's happier by the way of these groups on average 30s guys are happier yes I'm going to look forward for those who are not there yet um we can look across time here's age 13 to age 50 just frequency of words normalized to give sort of a t statistic a z-score how far it is what do people talk about school when they are young teens College then increasingly office and family as one gets older as my grades as one approaches the end of life and when it's 50 years 50 is not the end of life oh just for more faithful head drinks just for fun um 19 you get drunk in your 20s you have drinks and then increases against beer and at my age more wine it's nice okay but I promise not about well-being and psychology um I versus we really simple measure again 13 to 55 we in your teens a fair amount of we really low point at 2021 what do we talk about I I same thing is true for me then going up to 50s lots of we much less I people in their 40s and 50s are much more pro-social much more or outward oriented how are you doing we're doing well thank you I think so so one thing we we only at this point have a few years because it's Facebook and Twitter there's not that much history the only people that I can look at which I want to is to look at changes across the teen years I now have people going from 19 to 22 sort of range right so we can start to do some of those obviously I don't have 40 Years of yeah this thing in personal letters going back in the 17th century it's exactly the same patterns okay cool there we go um and it's not just I we the blue line going up friends family wonderful grateful and the low one going down Green stupid dumb hate smart idiots annoying what is smarter than uh um oh I'm not sure why the set of words I picked are sort of random I just picked a whole bunch of them and typed them in the results look very similar for all of these but again more positivity um here's some more of the more specifically going up being more grateful and proud less hating and bored and less excitement interestingly Peaks there and it's sad or not sad the fact of the matter is older people are less excited life is more exciting when you're young but you're more grateful and more positive later on so it's a trade-off enjoy the excitement now later on you're gonna be happy just to be calm and peaceful statistically you're allowed to but yeah again yeah right again we're averaging over large numbers of people here right these are all not individual these are all group statistics yep um that's interesting question and we have not done too much there are variations in personality by location across the us we've looked at a bit and there are certain variations on these I've not looked at these we should that's a great idea we should look at yeah I don't have actual income from these people we're trying to build some models to that that would be really nice what we do have is income at the average county level we could do that but you know all these we could do because these are all for all the Facebook ones we have location and for a fraction I'll talk about Twitter we have geolocation so we could look at a lot of these by income that'd be fun to do to do income too yeah I the literature suggested that this is fairly common across different income ranges as well but I'm I'm not sure that would be interesting so I'd be really interested in looking at different count that's a great idea I would look at that have not done it cool cool level modeling include by a set of studies looking at Community level modeling so we do is we take a couple billion tweets we geolocate them as best we can now about two percent actually have a coordinate that long no problem map them to County about 25 percent of them have some sort of a Twitter profile that if it says San Francisco and it's in the US it mostly is San Francisco or San Jose if it says Rochester who knows whether it's Minnesota or New York and a lot of them say my backyard or the world or my dog house and those we throw out so there's a lot of hacky code that Maps the tweets to geolocations counties but we get enough of them yeah so you throw out 75 too bad um there's plenty left right no shortage of tweets so we take the Twitter take the words same analysis same topics we take outcomes from the Centers for Disease Control and from the U.S census we know how happy people say they are we know how healthy they are and I think I'm going to focus time we'll go backwards focus on a hard outcomes we've looked at happiness reported and subjective well-being which you can see it's variation find the words but one interesting question is I claimed we started that happiness drives I implied health it's known that people who are depressed are at risk for cardiovascular disease it's known that being stressed is not a risk factor for cardiovascular disease heart attacks if you're running a startup and are stressed no increased risk of cardiovascular disease if you've got a crappy boss and you have no control of your life and life sucks and you're stressed you are at increased risk for heart disease so the modern theory is not the 50s type a thing which has been debunked but good stress if you feel control of your stress good no negative effect if you have no control over your life often from a bad boss the most interesting studies um you will on average die younger your mileage may vary so the question is we know that a bunch of of features of your environment like controller non-control affect your longevity can we actually go in and measure Twitter from a county and mortality in the same county now note this is a little bit weird because I'm going to show you arterial sclerotic heart disease dying of hardening of arteries there's also very CDC data we know what people died of the people dying are not the people tweeting to State the obvious the tweeting people are in their 20s and 30s and teens people dying are hopefully mostly in their 70s and 80s sometimes 60s not the same population so we're getting a set of words from a community and we're getting mortality from the community but they're different people so it's sort of amazing there might what is this evilness on my computer why do we want Java I've never had this problem um so here are the words that are used more in counties with higher arteriosclerotic heart disease I'll show you some clusters on them later um sexist is part of it seems potentially yep here are the counties that have lower mortality okay so what do we see here it's fun okay this is good this looks like San Francisco um what you see here is a whole mix of stuff a lot of these are socioeconomic status and some of these are actually more personality I'm going to tease them apart by going in and looking at topics the exact same latent dirichlet allocation topics in fact to be massively lazy I'll just reuse the ones I built on Facebook and use them on Twitter um so what do we see in counties I've shown you each case three of the ten biggest ones so pulling the ones that are most psychological out of the top ten counties with lower heart disease overcoming challenges opportunities ones with higher heart disease when Mike Ted trippy this is anger I think my psychology friends go that um so in addition to socioeconomic status and I will try and convince you this much more formally in 30 seconds there are psychological profiles of your community that seem to affect something correlate with heart disease so what do we know about heart disease we know that being depre individual level being depressed is bad more heart disease we know that anger angry men the cities on men tend to be a little bit more prone to heart disease our statistics are language much bigger effects angry counties your neighbors tweeting angry stuff [ __ ] [ __ ] you more dying negative emotions social isolation words the blue are the words disengagement bored tired lots of tweeting of bored and tired much more mortality for material heart disease on the protective side there's not a lot of individual level stuff on the psychological side I mean exercising is good of course we find positive emotions expressed in Twitter and social support and most importantly this we call engagement communities that are excited looking forward to things as opposed to ones that are bored and tired so one always worries that there's lots of socioeconomic status underlying these because there are but we know the top 10 socioeconomic correlates of heart disease what being um male is bad being until I can't say being black is bad being black leads to an increased chance of mortality for heart disease in the U.S right um negative things what's more Civic obesity hypertension diabetes and smoking this is again statistics counties with more obesity or more smoking or diabetes have more mortality foreign income and education actually more important than all of those counties with lower income and lower education of the two educations more important more heart disease put all those together we get a fairly High correlation with counties use Twitter alone we can do better with Twitter than top 10 demographic ones including you know demographics including diabetes and smoking right yeah so what are we capturing on Twitter we're capturing socioeconomic status it's there guess what educated people tweet differently from now I'm educated we're getting race guess what blacks and people of Indian background and they're tweeting differently we could tell these things apart but on top of that is still a personality profile and anger a disengagement a boredom or an excitement so that's interesting to me that we can profile personalities cool so that was the community piece and now I'm pretty much wrapped up I'm not going to talk about this but we're now going into hospitals and we've got the first thousand volunteers we've got their full Facebook posts and their full electronic medical record we're building models that predict oppression we can predict how depressed someone is as accurately from their Facebook as from the two question screen that we use in the hospital so take the ground truth the medical record depression you use the two questions screen use the Facebook we're about as accurate so we can now monitor that we're now going to start doing pregnant women who are at risk for depression can we monitor for changes in depression and give referrals to their doctors or their husbands if they have them and want them shared again huge privacy huge opt-in questions here but in general we find in the hospital not surprisingly someone comes up in a white lab coat with their iPad says excuse me do you use Facebook are you willing to share with us most people say yes because the iPad has their full medical record on it the triage information their things they're already giving blood samples giving a Facebook sample is not that painful right it's a trusted setting quite different from on the web will you share your you know that's a much lower piece there but in these sort of settings we get quite good response so we're building up these really nice pieces there I think it's part of a much broader Trend where I think of language again not as just facts about how do you feel about this product but language is reflecting your life and correlated with whatever I know about you right personality but in this case can I predict when you go to the ER we're going in we have funding we haven't done it yet to go into opioid treatment clinics and collect Facebook from people and correlate it with who completes or doesn't complete their addiction treatment I would love to know what triggers dried people to trivially go off their diet we're doing that one with a but more interestingly relapse in a drug use or recover I love to have more customized treatments based on people's personality it's insane that our Hospital you go in for an operation everybody gets the same mimeographed pre-op and post-op instructions regardless of education level if you've gone through conscientious or not I look at them I first of all it's paper I mean like to email English to me but quite apart from that the fact that the language is not customized the follow-up is not customized there's no accounting for outcomes of of who's calling whom or texting or tweeting so I think there's a lot of cases where having some notion of someone's personality is going to lead to much better treatment and much better responses so what do I take in at a high level it's striking to me how much the people around you influence your happiness that should be obvious but it's hard to measure if you're into the self-monitoring piece I think you'll see a lot of apps coming out or you can monitor not just how much you exercise how many nasty grams are you getting from whom what's the piece there I noticed there are companies out that are saying here's how to change your outgoing email message to make it more casual more formal more friendly more distant there'll be as customization not based on content but based on how does it make you feel um obviously these things are cheap and easy and you can get fairly good resolution especially as people treat from their phones new things surprisingly good right we said you could measure people's personalities accurately as their friends do that's sort of nice we can do depression as accurately as a standard quick screen done by a nurse in the hospital lots of stuff there and things I wouldn't have thought oh Sports could be good maybe you should try and treat neuroticism by suggesting people do more Sports I don't know you have to do a clinical trial but at least it's an idea or talk about sports I don't know right does the talking does it matter is the fact that it's a social activity that mostly Sports your way of bonding I don't know but at least it's a hypothesis that someone can go in and do the follow-up studies on um and of course the big team of psychologists and computer scientists so thank you all and yes questions people organizations education yeah so so for the Twitter we usually screen out anything which is a institution so if you've got more than 40 000 followers you're not really a person even if you're Oprah Winfrey so we do we typically screen out some piece there we've also thought about if I have not done his personality for Brands it'd be fun to do different corporate Brands how is an Android an iPhone different personality profiles um I don't have access to click-through data access to the likes and the pictures or no we so we can get likes but not click through so again all these things are Facebook and Instagram review board approved pieces you consent via US and via Facebook to share certain information and actually Facebook is really cracking down on what you can take and use they've gotten embarrassed a few times by studies that showed they were manipulating people's happiness so they've gotten really touchy about people downloading stuff so I'd love to have more on that but I they're a big privacy and political issues in the back and they'll come back to you uh yep go ahead awesome so you mentioned a couple of applications yeah let's see but do you have like a big dream application of This research that you'd like to see come to exist in the real world in in the short run I think apps like monitoring for depression are the obvious ones in the long run I guess I'm interested in a whole sort of Quantified Self style that I think we're mostly not that aware of how much the people interact with us influence our mood and behavior and to have more insight into what makes people happier and more successful as a self thing would be really nice so I'm in the positive psychology Center we're less interested in treating sick people and more interested in actually making well people better I think there'll be a lot of spin-offs IBM just launched last month their personality profiler so there's a bunch of there's a number of startups around here so a lot of companies are doing the same sort of thing HR companies are interested for hiring can you profile you want conscientious agreeable people I think there'll be a ton of business applications I'm happy to help people with them but my primary focus is more on helping to well-being the other thing at the county level I'd love to know the influence of political decisions on people's well-being not just positive negative sentiment but 20 000 people get laid off how does that reflect relationships there's a lot of we know that people are very unhappy even get laid off a year later you're back in work you're still unhappy after that but we don't really understand how it plays out in relationships we don't quite know for people recovering from cancer a lot of cancers like breast cancer most people get better but it's very disruptive to your life how was it disruptive so I think there's a lot of questions where there's a fair amount of relationship and there's increasing suggestion that people who have gotten good social networks recover better from lots of operations and diseases how do we provide that could we provide people have tried and failed in the startups to provide you here's friends you should reach out to and contact but I think it was an implementation issue not a fundamentally bad structure uh I just wanted to throw out that credit score would be an excellent socioeconomic measure if you wanted to like specifically either pull that out yeah or like if you wanted to use that as a predictor itself yeah Mark uh I just wanted to uh point out an interesting lesson from this work which is that it produces really spectacular results using really simple and rather traditional techniques so I'm not not to take anything away from the implementations and so on which have been to the point and clever and so on but the techniques that we're talking about are things like word counts Mutual information to find significant engrams like in the original allocation to find clusters of words and correlation yeah no this is amazingly dumb stuff word topics it's useful to have the engram the phrases with bush information gives much better quality it's nice to take the square roots of the counts to stabilize I didn't mention One technical issue don't correlate raw word counts take the spur of the counts and do massive control for overfitting right because everything appears to be significant if you're not somewhat clever but it's not really hard it's not rocket science it's really easy but psychologists never did it they don't know what to do with a billion tweets well also I think there's a tendency for people considering whether they're psychologists or computer scientists to feel that they have to do something very fancy yeah there's nothing wrong with fancy stuff but we've we've thought about parsing but the only real application here is is the psychological definitions of optimism and pessimism involve causal structures so pulling out causality and connection across different sentences is useful for some of these formal pieces but for very little let's see clear connection with uh there is a whole uh uh set of devices this is a very natural thing for humans a thousand years ago to you know turn on LCD or in this map I don't have a whole bunch of crap like pour into your head right absolutely right this is a recent activity which started more and more times yeah so it's surrounded by negative news and so the result of advice on NPR was to suggest like you should yeah particular television news yeah I think you could Facebook tried tweaking people's feeds a little bit and found small effects and got in real trouble for it and someone monitoring time by the way I don't want to run over my okay good but if if okay but if people want to go out I won't be insulted you know yeah because I'm having Coalition because yeah so this could be taken to the negative extreme and that there's this famous paper test about the serial killers so you could take that and you'd be deemed a serial killer even though maybe you are a so this is sparse enough to if it's taken as a predictor you might you could use this as a accusatory thing so so these things are all statistically significant correlations they are accurate at predicting you say your personality or male or female does it cause you to be male or female no does it cause your personality who knows does it cause the heart disease maybe maybe not by the way when I first did these I tried them at the state level got complete crap state has normally 50 observations but in fact they're massively correlated about 12 degrees of freedom across the U.S states it's not enough to actually capture any variation you get all sorts of it's called ecological fallacy for example regions that have more museums have more crime cities reads you with more phds in them have more crime cities so it's really easy to get spurious correlations whether the difference between urban and rural and only if you get a big enough sample size do you start to actually pull these apart seventy thousand people seems pretty good 2 000 counties which is the number that are actually real people in them they're a bunch of Montana that's really hard to do any data on um seem to be enough to get these but there's always this danger that you've got to be yes we find correlation not causality if you want causality go run an experiment but I'm happy for many things to know that and for a lot of stuff I don't care about causality you might want to know is this person hitting your site female or male young or old neurotic or well-adjusted great that's not a causal question that's a predictive one those are pretty good academic statistics before how did you uh ensure that the Norms of the Interior to ensure that the clustering seems so unreasonable wasn't biased to get what you wanted to see so the answers we build the same clusters once and we've used them on dozens of different problems we use them across different social media we've tried various mixes you can combine both Twitter and Facebook and co-cluster them and get Minor Details but it's important to note these LDA topics we're not doing them as supervise it's purely unsupervised do it once the only thing we've played with a bit for example with the Facebook medical records we will run LDA on medical records meta people only sorry the Facebook of people from whom we have medical records the population at Penn's hospital is mostly African-American poor very different base rate use of words than the us as a whole which is our Facebook sample so we do a little bit of it but mostly we just don't tweak things very much and the answer is if you're worried train on the first 10 000 people do the model the next 10 000 people really cool the question is do you want a hundred 500 or a thousand or two thousand topics yeah so that's number of topics matters and for the topics actually do use stop words for the regression and stuff I don't it turns out that many stop words are important old people use the more than young people so so things you think of the stop word normalization all a bad idea capitalization all these things are useful periods you know you guys are probably old enough informal if you often put periods at the end of your sentences very retro you might the old people but two spaces after a period so so again not a lot of customization it's mostly build a bunch of topics throw them in and run them and then we apply the personality models across so we're building personality models on Facebook and we're using them on Twitter not refitting them not readjusting them out of the box is Twitter language different than Facebook of course different distributions of emoticons and lots of things are different so what let me suggest One Direction Where We could get some useful practical result potentially and that will be a study of bipolarly disorder yes we've thought about that Peaks yes the oscillations that would allow because when that goes and diagnosed which happens it's yeah very bad and this is a disorder that's associated with a high level of drug abuse we have masks and symptoms we'd love to have it yeah yeah so I need to go find some study of it on our random sample of a thousand Hospital people there aren't that many bipolar unlike depression which is massively common it's easy to get a big collection of depressed people just walk down the street bipolar is happily much rarer so so small samples don't get large numbers of bipolar people so we need to get some more people to actually but we'd love to do that and we'd love to distinguish these very different sorts of unipolar versus bipolar depression obviously not the same but be interesting yes I'd love to do that if you if you have a sample of a couple hundred even but ideally 500 bipolar people send them my way preferably one to use Facebook Yeah you mentioned that there's some Services out there that allow you to they'll scrutinize and basically classify your your texts yeah just yeah give you yeah Personality yeah um we have do those Services also allow you to suggest changes no the ones that I know are mostly think of think of IBM they're helping large corporations do targeted marketing the theory is that if you're pitching a product different products will be sold to extroverts versus introverts and maybe of different ads with a bunch of faces around it and people doing it together for something to do alone so the idea of the of the first wave of these is very much targeted marketing rather than self-help or Improvement there's the immediate monies on marketing you're writing a cover letter you don't want it to sound bad yeah I haven't seen that yet yeah yeah more important question yeah um I was wondering like if you have a system like that I would think that um you know if you have a wide scale of use of that system suggesting to people how to change their language um would you see then a change in their behavior that's a great hypothesis we don't know I'd love to run experiments we will eventually where we actually get people to change their language Target intervention and a control group that doesn't and because some sort of of other treatment and see whether that changes their behavior or happiness I don't know that experiment as far as I can tell I've looked has not been run I would love to remind it oh yes that's a good good yeah a lot of good motto so yeah so we that's on the list to do but um we've done it we've not done it as far as I know no one has really yeah I fly Buddhism yeah sorry in the back of the ethics of predicting personal traits about another person so here we have the case of bipolar disease you have a few people who agreed to tell you that they are and then you build them all based on their social media presence and now you can predict whatever somebody's bipolar depressed without them giving them consent to do that so really this models in some senses I would also consider to be private information there's huge problems interestingly in the last year there have been two different cases of suicide prediction with very different outcomes in England there's a group that's been around for a hundred years or very reputable Suicide Prevention group and they put out a product a free product to monitor tweets and detect people who are potentially suicidal and after about three days of major Flack they had to retract it it was so controversial that people would be stalking other people with no consent monitoring their tweets and detecting who especially suicidal and these guys are really trying to help right but they had to withdraw the product two things on the other hand Facebook which has faced a lot of issues about their privacy is in fact doing something fairly similar monitoring for potential suicides on Facebook and providing interventions or contacts to Suicide Prevention organizations and as far as I know they haven't still running they haven't received any major Flack on that so the details of the Privacy is really subtle am I providing you potentially some information for you am I telling Dimitri hey that guy over there could be suicidal right depending on depending on what the opt-in procedure is and what's shared with whom it's very different and I think we're gonna have a whole bunch of controversy as to what where that line fits in terms of how much you're allowed to or it's appropriate to to find out about people so it's gonna be an interesting question I don't I don't know where the line's gonna be yeah let me argue a little bit I would suppose that if somebody stood out of here he recognizes of their suicidal but if somebody is bipolar he or she maybe I'm aware that this partition is actually bipolar disorder that can be treated and made me say that yeah they're all different um so there's a bunch of those cases I'm talking to a guy who's interested in in schizophrenia psychotic breaks and wants to start doing web Source things to figure out are people hearing voices not the court you want how much are they aware so there's a bunch of these things right yeah yeah let's get lunch I will be around for the rest of the day for questions thanks for the suggestions foreign