Charting Collections of Connections in Social Media with NodeXL

Bay Area AI: Marc Smith, Charting Collections of Connections in Social Media

Recording: Bay Area AI: Marc Smith, Charting Collections of Connections in Social Media

hello everybody welcome to well the sociological segment for today I am mark Smith I'm a sociologist I work with an organization called the social media Research Foundation and we're here in California in Redwood City and around the world as you will see shortly with our if I can find it here hang on a second there's our so we're here to talk about the social side of the social web in many ways the surprise of the Internet is that it became scaffolding for social relationships it was not intended to be a scaffolding for social relationships it was intended to make efficient use of expensive supercomputing compute cycles and instead what we have is a place where millions if not billions of people now flock gather and form relationships the people in the academic world who study the form of relationships are sociologists sociologists are social scientists who study a variety of social formations but there's a subsection of sociologists who study social networks and social networks are any network that have more than one human in them and so you're probably familiar with computer networks and we know about electrical networks financial networks and then there's also epidemiological networks who sneezes on whom we in sociology are interested in particular in social networks who communicate or have a relationship with them and in some cases those relationships get mediated sometimes I don't necessarily like or follow or friend you but you have a page or photo and that artifact acts as an intermediate but interestingly what happens online is that crowds form large groups of people gather and whenever large groups of people gather it's often considered to be newsworthy if I told you that hundreds gathered right now in front of San Francisco's City Hall that would be newsworthy and if I said they were protesting or they were in support of something that becomes very newsworthy whenever seven hundred and fifty thousand of your closest friends gather you can often make history or change it the issue for sociologists now is that crowds now gather in the virtual world more than in the physical world if I told you that hundreds if not thousands of people were gathering in front of San Francisco City Hall you would be surprised in part because that really doesn't happen that often it's very expensive for people to gather in the streets and so people gather online they tweet and they go to reddits and they have emails and Facebook posts and they have Instagram posts and all of this activity all this tweeting and posting and clicking and liking and following and friending this is the stuff of social media and it's hard to bring into focus it's hard to really say meaningful things about this stream because in many cases it's being looked at as if it was a bunch of individual messages or in some cases melted down into one big pile of words sociologists have a slightly different perspective on things we do not just see a bunch of messages or atoms we don't see individuals as really individuals frankly we're not the most mainstream American ideology out there you know people are not individuals they're actually the overlapping fields of their relationships and so what we're interested in is figuring out how to see the crowd mess in a online crowd how to recover that information because after all if a physical crowd in front of a politician looked like a tweet stream all of those people would line up single file and it would lose all of the crowd ness of it how big is the crowd is an angry crowd a frightened crowd a divided crowd what kind of crowd and I would argue as a sociologist that we are all sociologists some of us are just professional in that human beings by their nature make judgments about the activities of groups of other people and so can we find the crowd Ness in a tweet stream and the answer of course is yes yes we can these are the social network connections among a group of people tweeting about a particular topic it's the last President of the United States and what it shows is that the crowd is not a single crowd it was at this point in history at least two crowds and that these crowds for not because of their language use because we don't calculate it that way but because of their social connection patterns some people connect to some people more than others and so rather than being a social venue in which opposites clash although we hear a lot of stories about conflict online well we actually see are two isolated polarized but dense communities communities of people who share possibly a political orientation probably a political orientation but what they really share from a computational perspective this they replied and mentioned each other more than people on the other side of that divide and it turns out that in social network theory we have this idea or this observation it's called homophile II and you know this idea in English by the phrase birds of a feather flock together now of course in English we also have the phrase opposites attract and those two things contradict each other but work with me here birds of a feather flock together so the Assumption here is that if you reply or mention or talk to other people you're actually probably more in agreement with them than disagreement as a result we see these clusters form we form these clusters using some clustering algorithms we're using something in this case known as closet Newman more we offer in our software variety of them they're three that we are offering and what it finds is that there are these people over here in these people over here and they're more connected to themselves into each other and we have helpfully colored them red and blue but you could guess their orientation if you've been following along on the American political scene by looking at the words or the hashtags used most frequently by the people in that group so note the flow here first came connection then cluster then content so in many cases we first start with content then we go to a cluster so what I'm looking for here is some sense of who's who and if you're talking about let say Rubio and the GOP and the Tea Party you're on one side and if you're talking about taxing Wall Street and forward on climate you're probably another kind of person and so this is an illustration of the way that people tend to cluster into their opinion tribe and so our goal is make these kinds of maps easier for mere mortals I assume I am in a room of demigods you all are coders are you not any software developers in the room I guess would that like of course we're also for develops okay yeah so so most of us are not software developers myself included and as a result I'm really good at pressing buttons and if you have a button for me to press I can press that button the challenge we've had is that there was not that button to press the go get me a network analyze it visualize it do a content analysis and write me a report that but so we built that button there is a button we call that node Excel there's a button it sticks inside of Excel and when you press that button you get networks in the same way you might get pie charts and so the opportunity here is to bring network theory to the corporate desktop to make Network a first-class citizen of the desktop in the same way that pie chart and bar chart and scatter plot is and admittedly most people can't go past bar chart pie chart scatter plot usually taxes the graphical visualization capacities of most viewers and I am trying to sell you now the N dimensionally you know exponentially more complex graph but work with me here trust me the world is made out of the networks and the world is a complex place networks that come out of it they're complicated too but what we can do is reduce complexity into certain key observations that I think most people face with a graph want to get so this is just a note that our historical roots don't go all that far back in 1933 it's the first published mention in the New York Times of this thing network theory admittedly prior to that it was called topology and the mathematicians had their way with it since Euler I think was 17 when 1786 Euler does the bridges of königsberg problem and he invents topology as Swift as they are the sociologist a mere 225 years later are right on the case and apply it to social relationships and he starts by actually drawing the relationships in Shakespeare plays and then starts to use this method to figure out why was it that a rash of runaway behavior takes place at an all-girls school in the Upper West Side of Manhattan this in the early 1930s and so this methodology now has blossomed particularly since the late 80s when the IBM XT allowed even social scientists to get grant money enough to get a computer what we now have is a burgeoning industry of network tools and datasets and methodologies for doing all sorts of things including characterizing the culture of your favorite large technology company and I will note that this is a cartoon and not data but if you follow the link at the top it's a good cartoon so I spent 10 years at Microsoft I assure you that that has nothing to do with reality and I live near Oracle so the idea that the engineering department is dwarfed by the legal department is somewhat amusing so that's kind of interesting and I guess this is the apple of old where the red dot is mr. jobs so the idea is that you could actually capture some of the flavor the organizational structure the differences in the ways that groups of humans can connect because it turns out the humans are not atoms or they are they have a habit like atoms do of clumping up into molecules and perhaps the right unit of analysis for our understanding of humans is not at the atomic level maybe it's at the molecular level or even higher and so what we want to do is make these kinds of structures easy to get at and so we're inspired a bit by this device this is the technology that made photography a hobby this is the Kodak Brownie snapshot camera and over the course of the lifetime of it and its children before it went extinct around 2006 I think the the chemical camera pretty much died off as we know it but you know for about a hundred years it had a good run and it meant that you have pictures of your grandparents in their bathing suits you know we all do have these you know snapshot pictures of the day-to-day life of people all around the world because photography became really really easy so can we now do that ourselves but for a different kind of thing I'm obviously you can take pictures easily you get a disposable camera or maybe what we really want is to be a digital camera but it's worth noting that this also went extinct just recently nobody would ever buy one of those so what we want to be is this we want to be the digital camera someday on your phone and it doesn't take pictures of crowds it takes pictures of virtual crowds so they're out there the hashtags the group's the means the discussion boards we want to be able to go and take pictures of them with a button you press and then let it do all of the stuff metaphorically than a camera would do it's going to do white balance and it's going to you know figure out all of these different exposure meter issues in the same way you should be able to go and press a button and say go get me a network analyze it visualize it write a report about it email it to somebody and put it on the web for me and so this tool basically fits inside of a spreadsheet and it makes it easy for people who do not know that Python is not a snake or they only think Python is a snake that's it that's right there that's the way I wanted to put it and so there's a lot of those humans out there but all of us are in networks and the more you become aware of that you become well you begin to have that thing we call the sociological imagination and that's actually the title of a book by a guy named Searight Mills from back in the 50s and the 60s before he was hounded to death he was a sociology professor at Columbia and he wrote books like the power elite and another book called the sociological imagination in which he tried to convey this notion that if especially in the United States if there is this inclination towards psychological explanations somebody loses their job you figure well there's something wrong with that see right Mills liked to write about the idea of social forces when a hundred thousand people lose their jobs it's really hard to explain that by a hundred thousand psychologies you're looking for something at the sociological level so what we want to do is make it four easy for people to be more sociological in their thinking and to be more sociological what you have to do is think link when you can think link you start to think about edges you start to think about relationships between things rather than the attributes of a thing you start to think about the attributes of the relationships between things and so we want to encourage you to get a second monitor and with that second monitor spread everything out and be in an environment conducive to the exploration of collections of connections of these connected structures and this one is a Twitter network structure but they don't have to be from Twitter they could be from Facebook and they don't have to be from social media we have one user I'm very fond of discussing dr. Diane Klein dr. Klein is a professor of Greek antiquity at George Washington University at Klein with a CCL ine and dr. Klein has been using node excel to map the six degrees of Alexander the Great if you type that in the six degrees of Alexander the Great she's taken ancient Greek text and with a yellow highlighter figured out every sentence that said and then Alexander did something with somebody at some time and I believe she used photocopies because they really don't like it if you use the yellow highlighter in the original so it's unfortunate so it turns out though that she's collected thousands of these what you would probably call tuples you know who did what with whom when and where and within a few clicks that gets turned into now a diagram and a publication which you can google and that has actually led to the formation of the american historical associations historical network sub-group so historians now are mining all of these original texts the the one I'm most familiar with most recently is the Salem witch trials Network who denounced who as a witch she's a witch burner and so it turns out that it was actually a class struggle between the newly rising merchants family and the older established agricultural rich family and they were being eclipsed by the new family and they attacked by denouncing the female members of the opposing family and this can be seen in a diagram these diagrams are really hard to make if you're not a software developer ah but not anymore now if you can type into a spreadsheet lots of people around the world are now able to point this camera at their local issues because you know sitting here on the peninsula in San Francisco it's a particularly blinkered view of the world if it's not trending here or in New York maybe it didn't happen but when we have users from around the planet we're starting to see what does a hashtag map look like in Seoul or in Jakarta what's it look like when it's from Canberra what does politics look like in British Columbia versus in the United States and we're number one the most polarized social media discussions anywhere on earth right here in America we won't talk to each other more than anybody else won't talk to each other and so the only way to know that is by sending these cameras if you will where these sensors out into the environment around the world to encourage people to picture their own networks it could be your own personal network but it could be the hashtags or groups or other discussion spaces that matter to you so we're the SMR foundation social media Research Foundation and part of the way we support the growth in education around social networks is to make social networks something that you might see online much more often it's worth noting that web browsers don't really show you webs that they show you pages they really are page browsers we're going to try to claim that this is a network browser and we have a site called the nude excel graph gallery you can think of it as Flickr for networks but nobody uses Flickr anymore because it's done by Verizon now and so maybe we could call it Instagram for graphs so we'll think of it as Instagram and like Instagram it's a place for our users to upload their images and annotate them and there are a variety of them and I think that's an important point to be made that networks and social networks and social media networks are not all the same that like molecules they come in a variety of shapes and structures and their shape and their structure actually tells us something in fact the same set of atoms can form different kinds of molecules than behavior matically differently and so as you look at some of these images I hope that you'll be asking yourself the question what what does that mean but at this point in that process note at least distinctions differences there are not uniform patterns and some patterns do reoccur and so the goal is to start to generate so many of these images that we can start to say of any particular one oh that's of this kind it has those patterns in that structure over here there and the proportion of these kinds of people to those kinds of people that means something and so we are building a series of tools to make it easier and easier to actually get a picture of a hashtag or a Facebook group or the cache of ancient Greek documents that describe the life and times of Alexander the Great the key point is that so much of the world is made out of information about who did what with whom and when after all it's just metadata but with all of that metadata what we've had in some cases famously was a failure to connect the dots and what we need are better tools for dot connecting and to broaden or democratize the dot connecting tools so that more people can be dot connectors clearly all forms of social media form social networks if your social media does not form a social network it is not social media it might be digital media but it's not social media all social media encodes ties between human beings and so when it does that it also leaves machine readable data about it and this is a wonderful thing because in the old days sociologists ran after people with clipboards and pieces of paper and stubs of pencils and they would ask you hey do you know Sally in accounting and do you know Bob and marketing and do you ever talk to him about problems in the business and this kind of data collection works it's expensive it's slow and now you're slackening and you're yammering and you're chattering and you're emailing and you're posting and you're you know all of that just leaves footprints in the sand and it tells us all we need to know about who did what with whom where and when and that kind of data the idea of having a collection of connections is important it's really what the world's been made out of for years and years and years and yet the tools for getting at it thinking about it they're denied the most I have a daughter who's 16 just finished 10th grade and at one point brought home geometry homework and it said the word vertex I got excited and she said calm down dad you know it's not your kind of vertex and I questioned why shouldn't it be my kind of berta and my kind of vertex is the dot in a graph if a vertex is a thing that's connected to another vertex with an edge and why don't we teach kids about networks why do we teach ourselves and the population as a whole and my guess is that it would be good because once you really see the world as a network and once you do you can't stop you realize that damaged any part of the network is damage to all of the network and maybe that's a positive message so when you look at the web you can see that there are many kinds of times there's many ways that somebody authors this link or tie the association the bond with somebody else and if you think about it unless you are a software developer the average user mostly is doing this every favorite every follow every link like and reply rate review all of those things it's an edge and so what we want to do is see that edges are everywhere they're the universal data structure on all of these platforms and to then recognize that the way that they are the same are different might be in the way that they allow for people to author differently structured networks and this means that we could start to say well in what way is Instagram the same or different than Twitter in what way is one hashtag the same or different from another hashtag and not say that from a kind of qualitative and I'm not knocking qualitative but you know that ten minutes yes sir can do the challenge with qualitative data is that there is now more of it than a human may consume I think if you are trying to study the tweet stream related to American politics and you are not some kind of synthetic human you cannot read fifteen million tweets a day and really come away with some meaningful summary and so the machines have made it possible for humans to create more content than humans can meaningfully consume and so it'll just be their job to get us out of that problem and they can because a lot of that content can be compressed in some sense it can be distilled into certain kinds of structures and patterns and so once we see that it's really just made out of different kinds of edges they come in different flavors and directions and of course you probably recognize the tupple you know it's the thing with the thing at a time with a payload but getting that data structure to be safe for the non programmer and then to make it to that it's it's just a thing that you see in a spreadsheet you press a few buttons and why bother well because there's insight to be had because there is not a single pattern of network there's a variety of patterns in network and in the same way that you might ask a geographer well why do I want to see this map you call like the land why would I want to see a map of California you know there are valleys there are mountains there's a river or two and you might want to know where on that landscape you sit similarly we think it's interesting that there are a variety of patterns and these patterns are derived from Twitter and we do believe that we limit the claim to places like Twitter that have reply as an edge type but what this shows and you can follow the deeper stuff with our report about it with Pew but in this report we basically say look there are all these shapes and structures there's the hub and spoke pattern which essentially means the audience there is a bridge who is the the usually rare connection between otherwise disconnected groups and then the ever-popular Island who is typically ignored but an important part of our networks the people with zero connections are proof that you are a brand and so these different patterns then tell us something you can start to look at these and see there's the six kind so these are the six different that's different these are those six patterns shown before but now with real data and so this is what happens when Americans talk to themselves about tax policy which is to say they don't really talk to them selves they sort of have this us-and-them discussion this is what happens when it's a small town or a village in cyberspace everybody knows everybody these are the people who are community managers and this is what a brand looks like these are people who mentioned the handset the Lumia saying things like I'm thinking about getting a Lumia I got a Lumia I regret having gotten a Lumia they might say those kinds of things these are people who are talking about the previous First Lady of the United States a different pattern and then this the hub-and-spoke pattern this is when all the arrows point inward at the hub and that's what we think of as broadcast or audience but it's the opposite of the support pattern and this is where Dell tries to listen and tries to care and in this case the hub points outward at lots of people rather than the other way around so in this case you see a lot of people retweeting Paul Krugman because he wrote an article in The New York Times and in this case it's Dell listens trying to say hey send us your laptop again and we will fix it so there is a copy of this book floating around out there somewhere does anybody have the book in front of them it's the book around if you haven't seen it and you want to get a PDF copy of it you should put your name and an email address on that yellow sheet of paper and you'll get a PDF of it and I should say the names of my talented and remarkably handsome co-authors that's Derek Hansen up there at the top and he's a professor at Brigham Young University and in the middle ben shneiderman professor of computer science at the university of maryland and so that's our data flow that's what we do and we do it with a click or two and the point is to make it so that the users don't have to think too much about it instead they should think about what you get at the end which is insights into connected things and that's going to tell you things like who is most central who is the mayor of my hashtag it's going to let you ask questions like how does my hashtag or group or discussion space compare from a network perspective with my competitors so it's important to think that the same number of people can form dramatically different structures and that knowing how many people are how many posts you have is not enough and that you also want to know how they connect to each other so those are our six kinds we do not argue that there is not a seventh we we want to be sort of like social physicists and say welcome to the beginning of our periodic table of the social elements these are the structures we've discovered so far if you can find number seven we only request you no data to prove that seven is a naturally occurring and frequently occurring structure with that I think there may be a few minutes left maybe there will be some Q&A should we have Q&A there may be Q&A with that I thank you for your time okay and sure Thank You Molly at the same way I think I've got you know we have a time series back here somewhere that just illustrates that somewhere in there that you can take a look at it over time and find spikes and use in the client sliders that will allow you to let they narrow the network structure limit the edges displayed to only the ones that are let's say during a particularly spiky period and on the live client you could do that in the web presentation and static presentations not so I not as easily do they have a temporal pattern yes and that pattern is usually extraordinarily short-lived you know the half-life of a hashtag may be shorter than a new cycle so things come and go really fast and and then sometimes they come back remember such hits of yesteryear as a border crisis hashtag border crisis you know when there were people on our border about to come over the border and you know for that for a while that was like Shark Week and it lives and it grew and then it was gone and so a lot of these really have very very periodic or at least just their short pulses and they're gone the ultimate challenge in I want to look at the effectiveness of messaging or the time to do that how my body was how my to get where you could Jack get better but again quality yeah Alex EDA of course miss spelled that leg so what we would do is say well let's go look at with a jerry brown governor of state of california or I think it was a bee 586 which is the new covered California bill which essentially says if we can't have health care in America we will have it in California for all it's a very controversial term right a bee 586 you know that that's like we're going to do socialism in California you can type a bee 586 into this tool pull out the network of who's talking about it and you very quickly see partisan lines and you will see within those clusters who are the leaders of these partisans and then within that cluster what are the URLs that are most frequently mentioned by other hashtags are essentially marching with this hashtag if you are a you know anti-vaxxer or I think it was I can't remember the the Assembly Bill 286 I think the governor signed a bill a year and a half two years ago which said if you want to go to California schools you will be vaccinated and and guess what that was controversial and you could really see who was who and why who thought it was controversial and for what reason so yeah here's SB to 562 and here's Cal lag so we could zoom in on this hashtag this is Cal lag the California Legislature and so it's a hashtag for inside baseball kinds of political news junkies and it's illustrating the shape of the graph but it also will tell us who are the 10 most central people talking about California state politics at the Statehouse level those are those people hopefully we are providing you with a follow button because you know you might follow them and then the smart tweet button which allows us to suggest how would you talk to that person this is a very important person you might want to talk to them about what they care about we analyze their content and bring up two hashtags and two words that are salient to them which suggests it is not the final draft this is the first draft you you would use this as your prompt use these words in a sentence so with these tools I hope that you could see that there are some applications to political discussions or the study thereof as well I don't think maybe we have one more question can we know right there sir [Music] it's a good question I would argue that the last election cycle here in the United States has opened up an entire new area of network science research about disinformation and there is an enormous desire to fix fake news I come from a part of the university that thinks that truth left the room about 120 years ago and that there is no thing that is true not true however I think what we can do for you is offer the idea of information provenance which isn't to say is it true or is it not true but says who where did this idea come from and that I think might be something that Network science can offer us as a solution to that problem thank you oh actually appreciated [Applause] you [Music]