data.bythebay.io: Kyle Hailey, Importance of rethinking data visualization
Recording: data.bythebay.io: Kyle Hailey, Importance of rethinking data visualization
so hi everybody I'm Kyle hailey I'm super excited to be here I love data visualization and now my current job these past few years I haven't done much but I'm excited cuz I'm gonna be starting with Google working on visualizing data for their cloud performance so I'm a performance guy my main point in this presentation is like data visualization requires domain expertise and I see a pattern happening over and over again where companies especially hires up higher ups just assign a generic developer or generic product manager generic data scientists to look at data and maybe visualize it and they might not have any domain expertise they might be good at what they do great program manager a great developer a great data scientist but if they don't know the domain that they're doing the analysis for it can lead to lead to some problems so this presentation the bulk of its on a case from Edward Tufte second book how many people have not read the tough t books has everybody read top t Edward top these books at least one the first ones the best but this is from the second one but I think what's important to me is I'm explaining things simply especially when it comes to visualization should be simple and this is interesting because one of the earlier talks like this is this sort of battle between simplicity and complexity how much we show and how much we hide but you can't explain it simply I like what Einstein said you don't understand it well enough and Steve Jobs has some stuff to say about this too so when it comes to designing an interface what I see and this is happens with your with when the when the director says ok we need a we need dashboard for this information the developer goes and looks at it says okay this looks simple but as they start looking into it they realize is actually complex so Steve Jobs says when you start looking at a problem and it seems really simple you don't really understand the complexity of the problem so after you look into it after I look into it then I say oh it's complex okay well it's complex I'm going to create a complex interface cuz I have to show all this data I want to expose everything I don't want to hide anything and as Steve Jobs says then you get in if you get into the problem and you see that it's really complicated and you come up with these complicated convoluted solutions that's sort of the middle that's the part where most people stop and that's what I see in the industry but the complex solution is bad and that's what we're gonna see the beauty is in the simple powerful but it's difficult to get that simple powerful solution so Steve Jobs says but the really great person will keep on going and find that the key the underlying principle of the problem and come up with an elegant really beautiful solution that works so Steve Jobs says simple can be harder than complex you have to get hard you have to work hard to get your thinking clean and make it simple so my advice is to prototype and iterate as fast as you can another problem I see in corporations is they come up with design specs a maybe spend a couple months in design specs they implement it without iterating over that interface so here's a problem if I want to look at it so I'm a performance guys I'm gonna be biased towards performance if I wanted to design a performance dashboard how do I do it I mean looking at system I want to decide whether it's working well or not and if there's a problem what is that problem I doubt anybody here knows what this is anybody having an idea what this is this is a standard performance report from an Oracle database and as far as databases in the industry Oracle's way beyond everybody else and performance instrumentation it is the number one database for freaking out a performance problem but if I had to of this and I brought these reports in the meetings I've shown the people exact line where their problem the data I told them what the solution is for that problem and I is just glazed over and the meeting goes on for the next hour and people argue I bring in a nice visualization maybe a bar chart with a spike say look at that spike here's the solution everybody's like yes let's do it and so this is an Oracle database which i think is cool because at least it's shrink-wrapped this is from grin and brick I'm Brendan Gregg these are all the tools you can look at performance issues in a Linux OS at the geek in me loves this the educator me this strikes fear in my heart okay so that I'm going to move on to the example from Edward Tufte so I mean I don't know if everybody I don't know what this crowds like and my company some of the people weren't born when this happened but in 1996 I remember exactly where I was the day after this picture so this is January 27th 1986 they want to launch the spatial next day they had canceled it at multiple times there's an enormous pressure to launch it the engineers have built a solid rocket boosters the two Rockets on the sides they're in Utah they're going to launch this in Florida so it's got to be cold to mean colder than a previous flight so the engineers are frantically back data tik Florida sang do not launch the managers are like what's the problem we have to launch so they estimated launch attempt sure the next day of twenty nine degrees Fahrenheit the problem that the engineers wanted to communicate was on these solid rocket boosters that any cylinders in the seal between the cylinders of this type of rubber so as it gets colder that rubber gets harder it doesn't seal as well and when it doesn't seal that means a solid rocket propellant can burn through so what do they do this went on for hours the day before the flight so they started faxing information this is they sort of sent thirteen faxes and here's one of them now if people have the problem the Oracle performance report I think having a harder problem with this these three ovals that I've marked these are three different ways from naming the solid rocket boosters now what is this when they launch a flight the solid rocket boosters fall back in the ocean they go and retrieve the solid rocket boosters and look at them for damage so they've done this for all the previous flights so this is damage that they found on the solid rocket boosters from previous flights now they named them in three different naming fashions which is a problem they showed tons of information about the damage but what they want to communicate is not that they want to communicate that it's a correlation between colder temperatures and damage but as the engineers and this is the problem I want to talk about a little bit as engineers they want to show the exacting information they show detailed information which for me is a problem so here's another faculty sent and they sent a bunch of misleading information from my point of view if they want to show the complete picture they want the managers to understand everything in the red zone those were not actual flights those were test flight test launches in the Utah desert so they're horizontal so the pressures on a horizontal firing of these Sol rocket boosters is different from a vertical so it really should not be sending that information what they're sending this is what's called blow by damage which is not really what they're interested in either what they're interested in is called erosion where it burns through the rubber stoppers and what's really interesting about this is um okay the don't you show some next slide yeah so that bottom right that's the next day's flight so that hasn't happened yet just above it I circle of 58 and 75 so 58 degrees it looks like there's damage at the coldest flight they sent in 75 degrees the hottest flight and this causes another major problem so this looks like there's damage to the hottest flight and damage to the coldest flight but the engineers were clear this is their final summary recommendation the predicted temperature for the next day was twenty nine to thirty eight degrees Fahrenheit on the top right they say do not launch unless it's greater than or equal to 53 degrees Fahrenheit now this is where Tufte comes in so Tufte took the data the damaged data and graphed it so this is this didn't happen that night this happened later when Tufte did analysis now here's the data the basically visualization of the data they sent on the bottom as the temperature and on the y-axis is that the number of damage incidents so the question is if you saw this as a manager would you launch next day do you think that is there are something to worry about is there a correlation between temperature and damage I don't see one and one of the guys said I'd love this is the classic debate between I look at management the techies and the management techies are saying don't do it managers think we have to do it and then somebody says well there's damage or the coldest and hottest temperature what are you guys talking about so what did they do they launched and it was a tragedy so we can actually see the fire so that top red circle there's that's where burns through the cylinder into the liquid tank and exploded so is a national tragedy so what do they do they launch a congressional investigation so now so we go from NASA engineers like brilliant NASA engineers now we're gonna go to some of the highest pay lawyers in the country and what do they do they charge it like this so they this is what Tufte calls chart junk and I've circled in these little purple circles they mark the damages with different sort of a visual effects to show this different kinds of damage and they don't have it there's no legend to tell you what this damage is and what they should they show the temperature so the goal here's a show correlation between damage and temperature so they show the temperature but how the heck do I correlate that okay let's go back and look at what Tuffy said so this is the data that tuff that Tufte charted but let's change it let's see if we can make let's see if we can communicate our point better so one of the first thing I chopped he does is he retards it but instead of showing number of damage incidents I've changed the y-axis to be the actual extent of the damage so some of the damages are much more deep than others so we're rating on the y-axis on the extent of the damage now this is a huge thing one of the biggest probably biggest flaws is sending the information about the successful flights there were a lot of flights that had no damage and the engineers did not send that information and this is where I get into some arguments with engineers one of the arguments about this analysis is the temperatures of the solid rocket boosters are unknown these temperatures are from external weather reports so this geeky like Colonel engineers like but you don't know what the temperature of the solid rocket booster is but from my point of view this is good enough it's close enough and I find this a problem with engineering a lot they want to be super exacting but often getting the super exact information is not possible but instead of not showing any date at all let's show data that can be useful be accurate enough so the next thing we're going to do is mark the difference it's on the far the top right point is now blue I don't that's pretty visible it's a different kind of damage so I don't know if it's important or not but let's just mark it with a different color then let's normalize damage so at 70 degrees there were a lot of launches successful launches and their success in their launches with damage so I average out the damage success versus damage that's debatable too in my opinion is a good thing to do at this point we have some super cool information every flight above 75 degrees didn't have damage every flight below 65 degrees did and now we're seeing I actually see a pattern from this point of view it looks like there is increasing damage and this is what the engineers wanted to see but probably most important piece of information is still missing this is the known world outside of this box we don't have any data what did they want to launch the next day or the red X is would you have launched now I seriously doubt it so point of this is National Engineers failed so I mean these like super smart engineers the congressional investigators these high pay lawyers they've made even worse specializations in my opinion so data visualization is difficult but we don't do it right the can be devastating so visualization can be powerful and believe it or not in my lifetime I think this is no longer the case but when I was at Oracle trying to create a dashboard the engineers did not want graphics they're like I want the exact data and it took me about six months to convince them that graphics were good so you I'm in imagine a lot of you guys a lot of you know this data so this ants comments cortex is four sets of data X and wide and the cool thing about it they're very different sets of data but if we average it the averages are the same the standard deviation the same and if I'm gonna do computer analysis I'll probably do a linear regression the linear regression is the same so what's going on these data sets I know some people are pretty good textual parsers I'm I'm very I'm very weak at textual parsing but when it comes to graphics boom these are the same four sets of data I can see immediately what they're doing and then the amount of data we can consume I don't think I need to continue but this stuff is amazing there's over 3,000 counties United States so this is mortality due to cancer by county across the United States if I had this in a textual format and I was trying to correlate the data I can't even imagine trying to tackle that in a visualization I can immediately see patterns but visualization can also opt you scape data and be bad so here's a performance monitor a famous performance monitor for databases actually thrown by Dell computers now but people love this thing what it shows is the different components so the databases are pretty complex usually have processes that maintain the databases state and they have user processes that get data out run sequel queries and then we have a bunch of different caches and stuff so this is visualizing visualizing all the components inside a database and the idea you can see what's happening and if something goes wrong the color coded but this is super misleading like something will go red but we don't actually know the relative importance of it so like if I read some data off disk and it's slow I can do one read off disk it could take a second but my database might be fine so one seconds like it like an eon into computers so this way will flash red but that might not be a problem at all because I might been one i/o and the rest of our in the averages out to fine here's what I call throwing spaghetti at the wall and this is what I see in the industry right now somebody tasks the development team to go make a dashboard so they go talk to engineering what should I show engineers here's a list that important statistics developers like okay I'll show all that in a dashboard and from my point of view at storing spaghetti - well there's no intelligence in this well there is some but not much as a consumer of a product if I buy a database product or something I want their performance monitor to have intelligence in it I want it to guide me point out the problem and guide me to a solution I don't want it to leave me up to figuring it out it's another database performance monitor again it's throwing spaghetti at the wall and so I went on I went on to Google I said monitoring charts and so I was gonna look for some bad spaghetti at the wall charts this is Amazon's Cloud watch this should be cutting edge and actually super excited cuz I'm joining I'm joining Amazon it I my goal is to redesign this so imagine you're driving a car down the highway and this is your dashboard so this is what a DBA for local database this is what its gonna look like to them by normally telling one run once an hour do you want that or would you read this something like this now I'm gonna have to explain this a little bit but this takes all the information all the core information about importance of a database and this can be applied you need database or operating system this is for local database but what we have here on top is a load on the data so the question is first of all is there a big load on the database or not I can see it immediately now now I see the values going up and down but are they big are they small what's the relative value well we have something that gives us a relative value that's a CPUs so if I'm running on a computer I only have a certain number of CPUs and if I load is above the number of CPUs and I have a problem so couple things I see all this in second it's easy so that's the stop activity the server loads coming from top sequel top session and so the question goes to so that was just one database what if I had a real estate of a thousand databases what we can do is basically due to traumatic number what I call average active sessions this is like okay so how many processes are active and running this what that the load map shows so the bigger the bigger the territory is the more active processes there are and it's color-coded by where they're spending time so blue would be I'm spending a lot of time on IO green on CPU and so the databases that aren't very active our dolphin on the bottom right the ones that are really have a lot of activity I might need a tune on the top left so this is sort of cool to me I've been a DBA a lot of my life and what I hear daily in and day out is the database is slow and I'm just like it's not slow but how do I prove it it's been really difficult to prove now if I have this load chart on a database so the middle chart here that's the load chart there's nothing in it there's no load on this database the database is not slow because there's no load on it now maybe the developers can't connect to it I don't know that's the different issue but now I know the database is not the problem so what do you want do you want engineering data that's not nothing really wrong with engineering data in itself but if I'm trying to communicate information this could be problematic like we saw with the NASA case same with pretty pictures I think these off escape the data but if we can come up with clear simple graphics then we can communicate our ideally and powerfully so in summary sexual statistics are difficult in general to parse some people really good at it they're usually the exception pretty pictures are misleading and the goal is clear powerful graphics so graphics can add clarity but my main point here is it has to be domain expertise and ideally somebody who's been in the trenches with whatever you're dealing with that person has an insights on how to display the data so basically that was my talk so thanks for coming